Proactive customer care in a communication system

ABSTRACT

Aspects of the subject disclosure may include, for example, automatically detecting a service issue of a telecommunication system that may impact a customer of a telecommunication service provider of the telecommunication system, predicting, by the processing system, a future customer care interaction by the customer as a result of the service issue, initiating a resolution action of the telecommunication system for the service issue before the customer contacts a customer care agent of the telecommunication service provider, wherein the initiating is responsive to the predicting the future customer care interaction and modifying a component of the telecommunication system to improve efficiency of operation of the telecommunication system by reducing a number of customer care contacts by customers of the telecommunication service provider. Other embodiments are disclosed.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority to and is a continuation of U.S.patent application Ser. No. 17/066,778, filed Oct. 9, 2020. All sectionsof the aforementioned application are incorporated herein by referencein their entirety.

GOVERNMENT LICENSE RIGHTS

This invention was made with government support under CNS-1717493,OAC1738981, CNS1750953, and CCF1918483 awarded by the National ScienceFoundation (NSF). The government has certain rights in the invention.

FIELD OF THE DISCLOSURE

The subject disclosure relates to proactive customer care in acommunication system.

BACKGROUND

Communication system operators such as cellular service carriers areconstantly pushing the boundaries to deliver a positive, meaningful andunique experience for each customer (user or subscriber). The vastmajority of cellular network outages that impact customers at scale areproactively detected and resolved without waiting for customers toreport them. However, service degradation caused by individual customerprovisioning and device configuration errors still rely largely oncustomers to make the first move. To meet customer needs, anunprecedented number of traditional and digital channels have been madeavailable, as customer support can be provided over the phone, throughsocial media, and by online virtual assistants. While these omni-channelstrategies have transformed how customer experience is managed, many ofthese strategies are largely reactive. Communication system operatorstend to investigate and resolve these non-outage related individualcustomer service performance issues only after the customer initiates atrouble request

BRIEF DESCRIPTION OF THE DRAWINGS

Reference will now be made to the accompanying drawings, which are notnecessarily drawn to scale, and wherein:

FIG. 1 is a block diagram illustrating an exemplary, non-limitingembodiment of a communications network in accordance with variousaspects described herein.

FIG. 2A is a block diagram illustrating an example, non-limitingembodiment of a system supporting basic communication services, as wellas the typical work flow used to resolve customer issues, functioningwithin the communication network of FIG. 1 in accordance with variousaspects described herein.

FIG. 2B illustrates an exemplary number of care contacts for technicalissues received per hour for a two-week period and an exemplarydistribution of the time difference between the last observed failureevent in the network logs datasets and the time when the customercontacts care to report technical issues in the system of FIG. 2A.

FIG. 2C illustrates exemplary types of issues that customers reported tocare in an exemplary system over a sample 30-day period and percentageof data session failures among all the service issues observed fromnetwork logs in the system of FIG. 2A.

FIG. 2D illustrates an exemplary distribution of three resolutionactions taken by care agents to resolve customer care issues in thesystem of FIG. 2A.

FIG. 2E illustrates a framework design for proactive customer care inaccordance with various aspects described herein.

FIG. 2F illustrates an exemplary sliding window design for featurevectors in accordance with various aspects described herein.

FIG. 2G illustrates the feedback loop of the feedback model inaccordance with various aspects described herein.

FIG. 2H illustrates impact of a sliding window and window size onresults of the feedback model of FIG. 2G in accordance with variousaspects described herein.

FIG. 2I illustrates a comparison of results for three different modelsin the feedback mode of FIG. 2G in accordance with various aspectsdescribed herein.

FIG. 2J illustrates a comparison of results for three different modelsin the feedback mode of FIG. 2G in accordance with various aspectsdescribed herein.

FIG. 2K illustrates results for the feedback model in accordance withvarious aspects described herein.

FIG. 2L compares results for an experimental group and a control group.

FIG. 2M depicts an illustrative embodiment of a method in accordancewith various aspects described herein.

FIG. 3 is a block diagram illustrating an example, non-limitingembodiment of a virtualized communication network in accordance withvarious aspects described herein.

FIG. 4 is a block diagram of an example, non-limiting embodiment of acomputing environment in accordance with various aspects describedherein.

FIG. 5 is a block diagram of an example, non-limiting embodiment of amobile network platform in accordance with various aspects describedherein.

FIG. 6 is a block diagram of an example, non-limiting embodiment of acommunication device in accordance with various aspects describedherein.

DETAILED DESCRIPTION

The subject disclosure describes, among other things, illustrativeembodiments for automatic resolution of customer issues in communicationnetworks. Other embodiments are described in the subject disclosure.

Communication network operators such as cellular service carriers oftenutilize reactive strategies to assist customers who experience servicedegradation issues. That is, the operator or carrier receives from thecustomer a complaint or other indication about a failure in service andresponds accordingly. A customer care operation of the operator(referred to herein as “care”) identifies the nature of the complaint,determines a suitable resolution and implements a solution. This canrequire an undue amount of time to respond and create additionalcustomer complaints about the response.

As an alternative and improved model, a novel, proactive strategy hasbeen developed that monitors, troubleshoots and resolves device andservice issues, without having to rely on customers to first generatecomplaints. The new strategy seeks to improve customer experience andcare operation efficiency by automatically detecting individual serviceissues. Individual service issues are those that are non-outage related,such as when a portion of the communication network is not functioningproperly. The new strategy further seeks to predict customers who arelikely to contact care to report their issues, and proactively triggeractions to resolve these issues. Three machine-learning based predictionmodels are developed, and fully automated system is developed thatintegrates prediction models and takes resolution actions for individualcustomers. A trace driven evaluation is conducted based on real worlddata collected from a major cellular carrier in the US, and the novelstrategy is demonstrated to be able to predict customers who are likelyto contact care due to service issues with high accuracy. An embodimentis deployed into a cellular service provider network, for example. Otherembodiments, such as broadband access networks, media access networksand voice access networks could be instantiated as well. Field trialresults show that the strategy is effective in proactively resolvingservice issues, improving customer quality of experience (QoE), andreducing customer complaints.

One or more aspects of the subject disclosure include retrievinghistorical voice call information and historical data call informationfor a plurality of customers, including a selected customer, of acommunication service provider, wherein the historical voice callinformation and historical data call information comprises informationabout network access by a plurality of customers to a communicationnetwork of the communication service provider, and training a machinelearning model based on the historical voice call information and thehistorical data call information. Other aspects include applying currentvoice call information for the selected customer and current data callinformation for the selected customer to the machine learning model todetermine a likelihood that the selected customer will require customercare assistance by the communication service provider, and responsive tothe likelihood that the selected customer will require customer careassistance exceeding a predetermined threshold, initiating a correctiveaction on the communication network of the communication serviceprovider to reduce the likelihood that the selected customer willrequire customer care assistance.

One or more aspects of the subject disclosure include receiving care loginformation and network log information about communications between acommunication network of a communication service provider and aplurality of customers of the communication service provider, extractingfeature vectors from the care log information and the network loginformation, and training a machine learning model using the featurevectors. Aspects may further include providing current call informationto the machine learning model, wherein the providing current callinformation comprises providing information about current calls for aselected customer of the plurality of customers, wherein the informationabout current calls comprises current voice call information for theselected customer and current data call information for the selectedcustomer for an analysis window, receiving, from the machine learningmodel, information about a likelihood that the selected customer willrequire customer care assistance by the communication service provider,and responsive to the likelihood that the selected customer will requirecustomer care assistance exceeding a predetermined threshold, initiatinga corrective action on the communication network of the communicationservice provider to reduce the likelihood that the selected customerwill require customer care assistance.

One or more aspects of the subject disclosure include automaticallydetecting a service issue of a telecommunication system that may impacta customer of a telecommunication service provider of thetelecommunication system, predicting, by the processing system, a futurecustomer care interaction by the customer as a result of the serviceissue, initiating a resolution action of the telecommunication systemfor the service issue before the customer contacts a customer care agentof the telecommunication service provider, wherein the initiating isresponsive to the predicting the future customer care interaction andmodifying a component of the telecommunication system to improveefficiency of operation of the telecommunication system by reducing anumber of customer care contacts by customers of the telecommunicationservice provider.

Referring now to FIG. 1, a block diagram is shown illustrating anexample, non-limiting embodiment of a communications network 100 inaccordance with various aspects described herein. For example,communications network 100 can facilitate in whole or in part collectingnetwork log information about usage of all or a part of communicationsnetwork 100 and care log information about customer interactions withcustomer care representatives about device and performance issues in thecommunications network 100, determining a likelihood that any oneindividual customer will contact customer care about an issue, andaddressing the potential issue before the issue occurs and theindividual customer contacts customer care. In particular, acommunications network 125 is presented for providing broadband access110 to a plurality of data terminals 114 via access terminal 112,wireless access 120 to a plurality of mobile devices 124 and vehicle 126via base station or access point 122, voice access 130 to a plurality oftelephony devices 134, via switching device 132 and/or media access 140to a plurality of audio/video display devices 144 via media terminal142. In addition, communication network 125 is coupled to one or morecontent sources 175 of audio, video, graphics, text and/or other media.While broadband access 110, wireless access 120, voice access 130 andmedia access 140 are shown separately, one or more of these forms ofaccess can be combined to provide multiple access services to a singleclient device (e.g., mobile devices 124 can receive media content viamedia terminal 142, data terminal 114 can be provided voice access viaswitching device 132, and so on).

The communications network 125 includes a plurality of network elements(NE) 150, 152, 154, 156, etc. for facilitating the broadband access 110,wireless access 120, voice access 130, media access 140 and/or thedistribution of content from content sources 175. The communicationsnetwork 125 can include a circuit switched or packet switched network, avoice over Internet protocol (VoIP) network, Internet protocol (IP)network, a cable network, a passive or active optical network, a 4G, 5G,or higher generation wireless access network, WIMAX network,UltraWideband network, personal area network or other wireless accessnetwork, a broadcast satellite network and/or other communicationsnetwork.

In various embodiments, the access terminal 112 can include a digitalsubscriber line access multiplexer (DSLAM), cable modem terminationsystem (CMTS), optical line terminal (OLT) and/or other access terminal.The data terminals 114 can include personal computers, laptop computers,netbook computers, tablets or other computing devices along with digitalsubscriber line (DSL) modems, data over coax service interfacespecification (DOCSIS) modems or other cable modems, a wireless modemsuch as a 4G, 5G, or higher generation modem, an optical modem and/orother access devices.

In various embodiments, the base station or access point 122 can includea 4G, 5G, or higher generation base station, an access point thatoperates via an 802.11 standard such as 802.11n, 802.11ac or otherwireless access terminal. The mobile devices 124 can include mobilephones, e-readers, tablets, phablets, wireless modems, and/or othermobile computing devices.

In various embodiments, the switching device 132 can include a privatebranch exchange or central office switch, a media services gateway, VoIPgateway or other gateway device and/or other switching device. Thetelephony devices 134 can include traditional telephones (with orwithout a terminal adapter), VoIP telephones and/or other telephonydevices.

In various embodiments, the media terminal 142 can include a cablehead-end or other TV head-end, a satellite receiver, gateway or othermedia terminal 142. The display devices 144 can include televisions withor without a set top box, personal computers and/or other displaydevices.

In various embodiments, the content sources 175 include broadcasttelevision and radio sources, video on demand platforms and streamingvideo and audio services platforms, one or more content data networks,data servers, web servers and other content servers, and/or othersources of media.

In various embodiments, the communications network 125 can includewired, optical and/or wireless links and the network elements 150, 152,154, 156, etc. can include service switching points, signal transferpoints, service control points, network gateways, media distributionhubs, servers, firewalls, routers, edge devices, switches and othernetwork nodes for routing and controlling communications traffic overwired, optical and wireless links as part of the Internet and otherpublic networks as well as one or more private networks, for managingsubscriber access, for billing and network management and for supportingother network functions.

FIG. 2A is a block diagram illustrating an example, non-limitingembodiment of a system 200 supporting basic communication services, aswell as the typical work flow used to resolve customer issues,functioning within the communication network of FIG. 1 in accordancewith various aspects described herein. For example, communicationsnetwork 125 may include embodiments of system 200 for providing accessincluding wireless access 120 to a plurality of mobile devices such asuser equipment (UE) 202. The mobile devices can be any type of devicesincluding, for example, internet of things (IoT) devices. The system 200includes infrastructure elements 204 and provisioning equipment 206.Embodiments of the system 200 may include, for example one or moreradiotelephone systems such as cellular telephone systems. Theinfrastructure elements 204 include one or more base stations 208 and acore network 210.

Communication service providers such as wireless carriers and theircustomer care strategies heavily rely on reactive approaches to customercare. Such communication service providers typically provide customeraccess to various networks, including for voice calls and for dataaccess, such as to the internet and facilities accessible over theinternet. Customers typically access such networks using a device suchas a mobile phone, a tablet, personal computer or other equipped with acommunication interface and data processing components.

When trouble arises, customers conventionally have to take the firstaction and initiate a trouble report, such as via a care call, onlinechat, social media, etc., with the carrier in order for the carrier tobeing the troubleshooting and resolution process for individual wirelessissues. The customer care technology in the wireless industry has beenburdened heavily with costs and there has been a tremendous amount ofinvestment to have customers shift from high cost channels, such asspeaking with a care rep, to lower cost channels, such as using onlineforms, self-serve, frequently asked question pages (FAQ) and interactivevoice response (IVR) systems. The software, system and algorithms inaccordance with embodiments described herein offer wireless carriers theability shift from reactive strategies to proactive strategies, andidentify individual customers that are experiencing device or servicesissues and take resolutions actions, before the customer even noticesthere is an issue.

The disclosed system and method assists with detecting which customersare experiencing service issues and are likely to call into care. Thisis not trivial, as customers' behaviors vary across a wide variety offactors. For example, a customer can have a low tolerance for issuesimpacting voice calls, but have a high tolerance for data issues.Customers also are not experts on the network; wireless carriers do nothave much visibility into the device. Care agents have to translate thecustomer's general complaint to a specific service issue via aquestionnaire in order to identify what resolution actions to take. Thiscurrent solution does not scale well, and may result in very inaccuratedescriptions of issues which prolong the mean time to resolution. Thedisclosed system and method remove the need for this and canautomatically map a customer's quality of experience to a networkoriented quality of service metric, and then automatically take theappropriate resolution action.

Some wireless carriers have proactively sent short message service (SMS)messages to customers during large scale network outages in order toprevent customers from interacting with their customer care teams.However, this has not been accomplished for individual device issues.Wireline carriers employ such an SMS strategy and have also takenproactive actions on the customer's edge equipment, such as rebooting aWiFi router or gateway. However wireline carriers typically own theequipment and have much better telemetry from the customer's device,relative to the wireless industry.

In some aspects, the disclosed system and method analyzes historicalrecords of customers that have interacted with care through multiplechannels to obtain the quality of experience. The system and method thenanalyzes the customer's voice and data call detail records to identifythe quality of service, and utilizes machine learning technologies toperform the correlation and determines the likelihood that a customerwill call in. The system and method utilizes a novel feedback loop torecalibrate the probability for each customer over time in order toimprove the accuracy of the predictions. The predications are generatednightly, for example, and are provided to a control loop whichexecutives resolution actions on the individual devices determined to bemost likely to call customer care.

The system and method in accordance with these embodiments providessubstantial benefits. For example, the system and method in accordancewith features described herein improves the quality of service forindividual customers, and achieves a reduction in customer care calls,through the noted proactive approach. The system and method enablewireless carriers to fundamentally transform how wireless customer careis approached and can result in significant gains in operationalefficiency, while also improving the customer's quality of experience.For example, through the noted proactive approach, the volume of networktraffic required to identify and troubleshoot a particular customer'sissues can be substantially reduced. The resolution of the issue can beinvisible to the customer, initiated by the network remote from thecustomer before the customer even knows there is an issue.

Referring again to FIG. 2A, the system 200 may support severalcoexisting radio access network technologies. These technologies mayincudes so-called second generation (2G), third generation (3G), fourthgeneration (4G) and fifth generation (5G) technologies, as well as thoseto be developed in the future. Each base station 208 establishes radiocommunication with user equipment such as UE 202 in an area served bythe base station 208. Radio communication may be handed off from onebase station to another in the case of mobile user equipment.

In one example, 3G and 4G radio access is offered by a base station 208referred to as a NodeB and an e-NodeB, respectively. In the core network210, in the case of 4G networks, data service 216 is provided byServing/Packet Gateway, or S/PGW 214. Voice call service 215 is providedby voice-over-LTE, or VoLTE which traverse 4G NFs at IP MultimediaSubsystems (IMS) 212, or circuit-switched fallback (CSFB) that traversesthe legacy 3G circuit-switched (CS) network. The infrastructure elements204 form a set of cascading network functions (NFs). The infrastructureelements 204 permit communication of voice and data among respectivenetwork elements 204 and with other communications systems ofcommunications network 125 (FIG. 1).

The provisioning equipment 206 permits control and supervision of thesystem 200. The provisioning equipment 206 may include one or moredatabases, servers and other data processing systems, switches and userinterface devices to permit control and supervision of the system 200.In one example, the provisioning equipment 206 may include a HomeSubscriber Server (HSS) storing information pertaining to the UE 202. Inanother example, the provisioning equipment 206 may permit detection ofand response to customer care issues in which a customer or subscriberexperiences device or service issues that require resolution.

Communication network operators such as cellular service carriers striveto provide a reliable, high-quality experience for each customer,including users or subscribers. However, to date, many customer carestrategies have been largely reactive, relying on the customer to makethe first move by initiating a complaint or other interaction. Carrierstend to investigate and resolve individual customer issues only afterthe customer initiates a trouble request.

To increase cellular service operation efficiency and to improve eachindividual customer's experience, embodiments of the system 200 shiftfrom a reactive strategy to a proactive strategy when dealing withcustomer issues. Embodiments of the system 200 provide a novel,proactive framework that automatically detects service issues thatimpact customers' Quality of Experience (QoE) and predicts a futurecustomer care interaction as a result of these service issues.

The system 200 further triggers resolution actions to remedy thedetected issues before customers contact support agents. This not onlyimproves customer QoE by minimizing the impact of service issues, butalso increases operation efficiency of cellular service providers byreducing the number of customer care contacts and the subsequentinvestigation and mitigation process.

Solutions described herein focus on technical issues, meaning issuesinvolving user equipment (UE) and service related issues, that impactindividual customer QoE. Other customer issues such as cellular serviceplan upgrades and billing events may be beyond the scope of the system20. The discussion begins with a deeper understanding of these technicalissues reported by customers, and utilizes machine learning techniquesto identify network and service performance degradation signatures thatlead customers to contact customer support or customer service (referredto as “care” herein). Solutions described herein operate to correlatenetwork logs and customer care logs to automatically take resolutionactions for individual customers.

The discussion herein addresses the following questions: (1) What typesof device and network issues are likely to cause service performancedegradation that impacts customer QoE in cellular services? (2) Whattypes of service performance degradation are likely to drive customersto contact care and report their issues? (3) Which actions should beinvoked on customer devices and network servers to resolve customerissues? (4) How effective are proactive resolution actions in terms ofimproving customer QoE and increasing service provider operationefficiency?

The problem of identifying which individual customers are impacted byservice issues may be complicated by a number of factors. First, whileavailable data sources are massive, some data sources do not have thegranularity or latency desired to compose a complete picture ofindividual service issues across the end- to-end service path. Second,not all customers report service troubles. Only a small fraction ofcustomers contact customer care to report service failures. Furthermore,not all customers report technical issues at the moment they occur andinstead may wait a period of time that is highly variable. Third, theservice issues that customers report to care can be ambiguous orinaccurate. The issue types recorded by customer care agents can behighly subjective, and it may be difficult to attribute them to specificroot causes. Expanded service offerings, new devices on the market, andmultiple resolution actions exacerbate the problem.

One or more of the following are addressed: (1) An automated frameworkto identify individual customers experiencing service degradation due totechnical issues, and resolve their issues by proactively invokingresolution actions, is described. (2) Data sources available to cellularservice providers to understand customer behavior are analyzed. (3)Novel machine learning-based models are developed to predict individualcustomers who are likely to contact care due to non-outage relatedservice issues using a combination of customer-perceived andnetwork-observed metrics. (4) A trace driven evaluation of the proposedframework is conducted based on large scale real world data collectedfrom a major cellular service provider. (5) An embodiment of theframework is deployed into the major cellular service provider networkas a field trial. Field trial results show that the framework iseffective in proactively resolving non-outage related service issues,improving customer QoE, and reducing customer complaints.

Issues that impact a specific user's session across the control plane oruser plane of a communication network may occur anywhere across thenetwork topology, and are captured via network fault and performancemanagement techniques. Even with this information, customer careorganizations employ reactive strategies when dealing with individualcustomers. That is, customers usually have to initiate a trouble reportto generate an investigation.

Referring again to FIG. 2A, the bottom portion of the drawing shows atypical customer care investigation flow. The flow in this exampleincludes three steps. In a first step 218, the customer contacts thecare center operated by the network operator or service provider and isdirected through an automated interactive voice response system to anappropriate first-tier team. In general, the first-tier team includeshuman personnel operating in conjunction with network equipment, such asthe provisioning equipment 206.

In a second step 220, the first-tier team performs a number of routinetroubleshooting and resolution actions. Such actions may includechecking customer account and billing status, service provisioningstatus, known outages and hardware/software issues. Generally, theactions are organized based on the issue type reported by the customer.Example issue types include billing, administration, technical, forexample. If the issue is not easily diagnosed and resolved, the issuereported by the customer may be escalated to the second-tier technicalteam for further investigation of network and service logs, along withperforming additional resolution actions in a third step 222. Thesecond-tier team generally also includes human personnel operating inconjunction with network components. The second-tier team may havespecialized knowledge or experience that is of particular value.

In the third step 222, resolution actions are taken to solve the problemreported by the customer. In general, all actions are automaticallyrecorded during this process. The customer care agent who is responsiblefor this customer-reported issue also provides a written summary,including the troubleshooting steps or resolution actions, taken andtheir results.

Customer care agents may handle a large variety of issues, which aremanually classified into a very large number of categories. Customercare issues are roughly divided into two categories: technical issuesand non- technical issues. A technical issue means that it is likely aresult of technical or configuration issues in the network or a deviceoperating on the network. Some examples of technical issues include,“Unable to make/receive voice calls,” “Unable to connect to dataservices,” “Cellular Data Connectivity,” and so on. Non-technical issuesare related to routine customer engagement or information inquiries,such as inquiries about new services or plans, service activation ordeactivation, billing-related inquires and device setup. In general,technical issues are relatively more difficult to resolve and heavilyimpact customer QoE. Accordingly, particular attention is paid to theautomated resolution of technical issue.

To improve responsiveness and reliability, the conventional reactivestrategy is replaced with a proactive strategy that automaticallydetects and resolves issues in order to reduce the resolution time andimprove customer QoE. The system and method in accordance with thisstrategy detect performance degradation issues faced by a customerthrough leveraging network, service, and customer care logs collected bythe network operator or cellular service provider.

As illustrated in FIG. 2A, there are available several data sourceswhich include data collected and aggregated from existing networkinginterfaces. The main data sources used in this example can be summarizedas follows.

A first data source includes care logs (CL) 224. Care logs 224 recordinteractions between customers and care agents. Each respective care logof the care logs 224 generally includes (1) user identificationinformation, or UID of the customer associated with the respective carelog, (2) timestamp indicating date and time associated with therespective care log, (3) care contact channel including online chats,phone calls, and store walk-ins associated with the respective care log,(4) issue type, which is manually provided by the interacting careagents and is associated with the respective care log, (5) descriptionof the issue associated with the respective care log, which may be infree text format added by the customer care agent, and (6) sequence ofactions taken for troubleshooting and resolution of customer issues.Conventionally, the care logs 224 are collected after obtainingcustomers' permission.

A second data source includes customer account information (CAI) 226.Customer account information 226 contains information regarding thecustomer service subscription. Examples include a user identifier, whichmay be anonymized, the device manufacture and model of the UE 202 of thecustomer, information about a hardware and software version, anactivation time, and information about a last update to the account ofthe customer.

A third data source includes network logs (NL) 228. The network logs 228contain information about how a customer device such as UE 202 uses eachdata service and voice service over the network. In some examples, thenetwork logs 228 consist of two datasets. These network logs 228 includevoice call records (VCR) 228 a. Voice call records 228 a recordinformation for each respective voice call. Data is collected, forexample, by the IMS for VoLTE calls, and by the 3G CS network elementssuch as IMS 212 for CSFB calls, for example. Table 1 lists exemplarymain attributes of voice call records 228 a. Typically, the voice callrecords 228 a dataset covers both successful voice calls and callfailures. In case of call failures, additional information may berecorded to capture the status and cause of record closing.

In addition to the voice call records 228 a, the network logs 228 mayinclude data connection records (DCR) 228 b. The data connection records228 b contain information for each data service. The data connectionrecords 228 b are collected from the S/PGW 214 for each packet datanetwork connectivity (PDN) session and may be organized by its AccessPoint Name (APN), for example. Table 2 lists exemplary main attributesof the data connection records 228 b. In some examples, in case ofpacket-switched (PS) calls (VoLTE), a predesignated APN may be used totunnel a voice call from the PDN network to IMS, and the sessions arerecorded in the DCR 228 b dataset. If a data session fails, the cause oftermination is recorded as a CFT code in the DCR 228 b.

TABLE 1 Attributes in the VCR dataset. Attribute Description UID Useridentifier (anonymized) Start Time Time the when Charging CollectionFunction (CCF) started session End Time Time when the CCF terminatedsession Cause for Reason for the release of the session. Record Closing(0) for successful sessions Status Abnormal status information of thesession. (if applicable) SIP (4XX/5XX) code (Blocked/Dropped)

TABLE 2 Attributes in the DCR dataset. Attribute Description UID Useridentifier (anonymized) Start Time Time when the PDN session starts EndTime Time when the PDN session ends CFT Code Cause for termination (CFT)for a PDN session APN ID APN name of failed PDN session

It should be noted that all the datasets may be indexed according to ananonymized user identifier (UID), which is different from the unique,permanent customer identifier such as an International Mobile SubscriberIdentity (IMSI) or phone number, to protect customer privacy. Theanonymization is conducted in a consistent way such that each anonymizedUID uniquely identifies a mobile customer, without revealing anypersonal identification information.

The current approach to automating customer issue resolution appliesmachine learning techniques to proactively identify customers who areexperiencing a degraded QoE, and to predict if they will contact care inthe near future. This problem can be modeled as a classification problemusing the features extracted from datasets collected by the serviceprovider. If a customer is predicted to call care, the system and methodtake action to resolve the issue, which will reduce the severity andduration of customer QoE degradation, and eliminate the need for thecustomer to contact care.

While determining if a customer is experiencing service qualitydegradation sounds simple, there are several technical challenges thatare inherent in the datasets used in the study. These are discussedbelow.

Data scale and quality. The datasets described above were collected froma major cellular service provider operating in the United States overseveral months. Developing models that generate actionable results forindividual customers requires data sources that provide UE granularinformation with a low to medium latency. Furthermore, an analysis ofthe network logs 228 datasets (VCR 228 a and DCR 228 b) for failuresrequires significant computational resources due to the massive volumeof the network logs 228 datasets. In one example, the network logs 228data includes over 100 billion entries per day.

Not all candidate data sources meet these requirements, which limit theability to capture the metrics from the end-to-end service path ofcustomers' cellular service. For example, as shown in FIG. 2A, completedata is not available from the radio layer interfaces, such as thee-NodeB, where many failures occur but are not visible from the corenetwork 210. To gain a better understanding on this data limitation, thecare logs 224 dataset and network logs 228 dataset were correlated, andthe percentage of customer-reported issues in the care logs 224 datasetwas computed that can be observed from the network logs 228 dataset. Itwas found that less than 25% of reported issues can be observed in thenetwork logs 228 dataset, and it is believed that the primary cause ofthis limitation is due to the incompleteness of available datasets.

Behavioral aspects in issue reporting. By correlating the care logs 224dataset and the network logs 228 datasets, it was found that only a verysmall percentage (0.3%) of customers who experience some kind of servicedegradation, such as a voice call drop, contact care. In addition,customers who experience performance-related issues do not necessarilyhave a higher probability of contacting care and reporting their issues.Reviewing data about the percentage of voice calls made by customersduring a time period of two weeks and the percentage of voice callfailures experienced by them, it was observed empirically that a largepercentage of customers who call care experience few voice callfailures. A similar pattern was seen when comparing the percentage ofdata sessions initiated by customers and the percentage of dataconnectivity failure events experienced by the customers. It was foundthat customers who contacted care do not encounter a large number ofdata connectivity failures; in fact, a large number of customers whocontact care encounter few (if any) data connectivity failures. Theseobservations imply that threshold-based prediction models cannotreliably be used to predict customer behavior.

In addition, differing customer attitudes may impact when and how oftena customer contacts care. FIG. 2B illustrates an exemplary number ofcare contacts for technical issues received per hour for a two-weekperiod and an exemplary distribution of the time difference between thelast observed failure event in the network logs 228 datasets and thetime when the customer contacts care to report technical issues in thesystem of FIG. 2A. Referring to FIG. 2B(a), this figures shows thenumber of care contacts for technical issues received per hour for atwo-week period. The y-axis scale is omitted due to preserve dataconfidentiality. As seen from FIG. 2B(a), care contact volume is oftenhigher on Mondays and Tuesdays, whereas weekends typically have thelowest volume of care contacts. Nearly 25% of care contacts are receivedfrom customers who have contacted care more than once a month. Nearly 3%of care contacts are received from customers who have contacted care atleast 4 times a month.

FIG. 2B(b) shows the distribution of the time difference between thelast observed failure event in the network logs 228 datasets and thetime when the customer contacts care to report technical issues. It wasfound that, among all customers who contact care, only 18% of customerscontact care within 24 hours of experiencing an issue, and 27% ofcustomers contact care after two days of experiencing an issue. Theduration after which a customer calls care depends on when theperformance issue is encountered (e.g., the day of the week and time ofday) and the failure patterns or signatures. In addition, differentcustomers likely have different tolerance levels to performancedegradation. For example, most customers may choose to ignore temporaryperformance degradation unless the issue is persistent or chronic, whilesome individual customers tend to contact care every time they spot aperformance degradation. Predicting customers who are likely to contactcare due to technical issues therefore requires understanding therelation between network-observed metrics such as Quality of Service(QoS) and customer-perceived QoE. Since QoE is affected by individualcustomer behavior and attitude, developing models tailored to improveindividual customer experience is a challenging task.

Issue classification and resolution fidelity. When a customer doesreport a service issue, the issue types recorded can be highlysubjective and sometime ambiguous. While service degradation can occuracross the end-to-end service path between the UE 202 and the corenetwork 210 (FIG. 2A), individual customers are seldom aware of the rootcauses. Given the complexity of the network, along with emergingtechnologies and changes in the device ecosystem, the actual categoriesof errors reported to care by customers are very broad. For example,some of the most commonly reported issues include “Unable to make voiceCall” and “Unable to connect to internet.”

FIG. 2C illustrates exemplary types of issues that customers reported tocare in an exemplary system and percentage of data session failuresamong all the service issues observed from network logs in the system ofFIG. 2A. FIG. 2C(a) is a pie chart showing types of issues thatcustomers reported to care over an exemplary 30-day period. It wasobserved that voice call and data connectivity issues constituteapproximately 93% of technical issues reported to care. While 54% ofcustomers report voice call-related issues to care, voice callscontribute only 1.7% percent of the total cellular traffic (voice plusdata traffic). FIG. 2C(b) is a pie chart showing that among all theservice issues observed from network logs 228, 91.7% are data sessionfailures. Customers are much less concerned about, or aware of dataconnectivity issues, compared to voice call issues.

In general, human care agents are trained to follow a pre-definedprotocol during their interactions with customers. Due to thesubjectivity and ambiguity of issues reported by customers, there may beinconsistency in diagnosis and resolution actions performed by suchhuman care agents. FIG. 2D illustrates resolution actions taken by careagents to resolve customer care issues in the system of FIG. 2A. FIG.2D(a) is a bar chart showing an exemplary distribution of threeresolution actions taken by care agents to resolve issues in thecategory “Cannot Make or Receive Calls.” FIG. 2D(b) is a bar chartshowing an exemplary distribution of three resolution actions to resolveissues in the category “Cellular Data Connectivity.” While the specificresolution action name are omitted for confidentiality, these resolutionactions include in one example triggering a network initiated detachprocedure, forcing the customer device to re-initiate the radioconnection, and flushing authentication vectors from the Home SubscriberServer (HSS).

Correlating network events to customer QoE. The network logs 228datasets may contain information about many events and alerts. There areseveral reasons why it may be difficult to map these events and alertsto a degraded customer QoE. First, the end-to-end service path of acellular service request usually contains multiple network elementsinvolved in processing cellular traffic. The following insights may beused to select features used in the design of a prediction model. First,most customers only experience either voice or data connectivityfailures, but not both bookmark 13. Second, customers have lowertolerance for voice call failures and therefore customers experiencing ahigher number of voice call failures are more likely to report issues tocare. Third, customers who make a large number of voice calls are morelikely to contact care and report issues. Therefore, it may be concludedthat customers who make a large number of voice calls and experience agreater number of failure events (QoS) are more likely to contact care,which results in a higher percentage of voice call-related issues (QoE)reported to care. A model to predict customer care contact behaviorshould thus focus on customers who (a) make a high number of voice callsand (b) experience a higher number of voice call failures.

Design Guidelines. The network logs 228 dataset includes severalattributes that are not useful in predicting user behavior, such aserror codes and network elements, some of which may be external to thecellular carrier network. Example external elements include elementsinvolved in processing cellular traffic. The following insights may beused to select features used in the design of a prediction model. First,most customers only experience either voice or data connectivityfailures, but not both. Second, customers have lower tolerance for voicecall failures and therefore customers experiencing a higher number ofvoice call failures are more likely to report issues to care. Third,customers who make a large number of voice calls are more likely tocontact care and report issues. Therefore, it may be concluded thatcustomers who make a large number of voice calls and experience agreater number of failure events (QoS) are more likely to contact care,which results in a higher percentage of voice call-related issues (QoE)reported to care. A model to predict customer care contact behaviorshould thus focus on customers who (a) make a high number of voice callsand experience a higher number of voice call failures.

FIG. 2E illustrates a system 230 for proactive customer care inaccordance with various aspects described herein. The system 230 is aframework for automated monitoring, detection, and resolution ofcustomer issues related to service performance degradation. As depictedin FIG. 2E, the system 230 includes two phases: (1) an offline phase 232in which historical data is used to train prediction models, and (2) anonline phase 234 in which the predictions made by the models are used astriggers to take resolution actions to repair issues impactingindividual customer devices.

As will be described in more detail below, the offline phase 232consists of three steps or operations or processes. First, the threeinput datasets (CL 224, CAI 226 and NL 228) are preprocessed in apreprocessing operation 236. One goal of this data preprocessing is toremove data related to devices or conditions that are out of scope ofthis study. Second, a feature extraction module 238 is applied on eachinput dataset. Third, machine learning model or prediction model 240 aretrained in a training dataset 242 for predicting customers who arelikely to contact care and report service issues based on the extractedfeatures.

In the online phase 234, the prediction model 240 takes real-timenetwork logs 228 as input, performs a preprocessing operation 244,detects failure signatures, and a predictions 246 predicts the customerswho are likely to contact care due to service issues. In a resolutionaction 249, resolution actions are then triggered to resolve theseissues. Service performance metrics are monitored before and after theactions to assess the impact of resolution actions.

It may be noted here that components of the system 230 can be deployedin any suitable location, or at a plurality of locations. For example,components that perform the functionality of the system 230 can belocated in either centralized or distributed fashions. In someembodiments, the system 230 must analyze a very substantial volume ofdata. For example, the system may need to process, for every subscriberor customer, that customer's network logs including call records andinformation about how that customer accesses the communication network.In some embodiments, components that implement the functionality of FIG.2E, including offline training and online processing, can be implementedat a single centralized location. That provides a benefit in that alldata for all customers is centrally located and available for processingby the system 230. A downside to the centralized implementation is therequirement to move all customer data to that centralized location,where customers and network equipment are very decentralized and spreadover a possibly very large geographic area. That can require substantialtime to push data from various data centers and convey the data to thecentralized location.

In an alternative embodiment, the offline phase 232 and the online phase234 of the system 230 can be deployed independently in separategeographical locations in some embodiments. This is indicated by theindices Offline(1) . . . Offline(n) and Online(1) . . . Online(n). Inone example, n=11 and eleven data centers are used for processingcustomer data. The model of the system 230 can be deployed at any numberof data centers, such as data centers which are physically closer toselected customers and the data that the customers generate. One benefitof such a distributed system is faster accumulation of customer data ina data center, faster operation of the model and a faster predictionproduced by the model of the system 230. Customer data can be broken upor organized or arranged according to any useful criteria, such as bygeographical range (all customers in New Jersey, for example), by userequipment (all cell phones produced by Apple Inc.) or others.

In the offline phase 232, training or testing sets are created based oninformation from the care logs 224, the customer account information 226and the network logs 228. In one embodiment, two training and testingsets are formed. A first training set 248 is formed by identifying allUIDs that have contacted care during a specified test period. Suitabletest period durations may be, for example, 6 days, 25 days, one month orsix months. A second training set 250 is formed randomly sampling UIDsfrom the network logs 228 dataset. The first training set 248 and thesecond training set 250 are combined to form a set of training useridentifiers (UIDs) 252. The set of training UIDs 252 form the input tothe preprocessing operation 236.

The preprocessing operation 236 preprocess datasets including the set oftraining UIDs 252 to filter out data in the three exemplary cases.First, the preprocessing operation 236 removes data records that arecollected during the time period when there were known data qualityissues, such as e.g., missing data, bogus data. Suspect data can haveundesirable impact on the prediction model. Automatically andsystematically cleaning data has always been a challenge and is itselfan open research problem. In one embodiment, the preprocessing operation236 uses a rule-based mechanism to identify and remove suspect databased on domain knowledge.

Second, the preprocessing operation 236 filters out all the data recordsrelated to known network outages in both the care log 224 dataset andnetwork log 228 dataset. Individual service degradations which are notattributed to network faults in some examples account for 95% ofcustomer reported issues in exemplary data sets, and are a much harderproblem to diagnose and resolve at scale, when compared to servicedegradation stemming from network outages. The system 230 focuses on thenon-outage related service degradation issues that impact customer QoE,such as, for example, device misconfiguration, provisioning errors,device software/hardware issues.

Third, the preprocessing operation 236 discards all roaming mobiledevices from the network log 228 datasets. A customer mobile device isdefined to be a roaming device if the customer is not an authenticatedcustomer based on the customer account information 226 data and thesystem 230 is not able to conduct any resolution actions on thesedevices.

The feature extraction module 238 generates usable feature vectors fromraw data. In the illustrated embodiment, there are two types of datasources: (a) static data sources such as the customer accountinformation 226 for which the information can only be changed by anetwork operator such as cellular Operations and Management (OAM), and(b) dynamic data sources such as the care logs 224 and network logs 228which change as customers use services or contact care.

Table 3 shows exemplary static feature vectors extracted by the featureextraction module 238 from the customer account information 226 datasetfor each unique device. The feature extraction module 238 in oneembodiment uses one-shot-encoding to convert descriptive features suchas “Device Information” to binary values such as “True” or “False.”Similarly, the dynamic records in the care logs 224 dataset and thenetwork logs 228 dataset may not be directly usable as features. Thefeature vector extraction module 238 calculates key performance metricssuch as, for example, usage and failure, using a fixed time bin asdynamic feature vectors as listed in Table 4. Examples of the timeduration for the fixed time bin are one day or one hour. Other timedurations may be used.

TABLE 3 Static Feature Vectors per UID Feature DescriptionActivation-Time Account activation timestamp device-type-appleTrue/False device-type-others True/False True/False

TABLE 4 Dynamic Feature Vectors per UID Feature Description #Blocked-Calls Number of calls blocked # Dropped-Calls Number of callsdropped # Calls Number of calls made # Call Duration Duration of callsmade # Data-Session (F) Number of failed data sessions # Data-Sessions(S) Number of successful data sessions # Care-calls (T) Number oftechnical care contacts # Care-calls (O) Number of other care contacts

In some instances, straightforward feature extraction may have somelimitations. These may include the following. A first limitation is thepotential existence of imbalanced classes for training. It is known thatonly a small fraction of customers experience service issues, and only asmall fraction of these customers contact care. This represents animbalance in class size for training a model. Such an imbalance istypically solved either by under-sampling the majority class in amini-batch or re-weighting its loss. However, in the current example,availability of fewer samples of the minority class, or customers whocontacted care, will require accumulating training data over a longerduration for a statistically significant balanced training sample. Asecond limitation is seasonality within data. Customer care centersexperience higher call volumes on certain days of the week, asillustrated in FIG. 2B(a). A third limitation includes temporaldependencies among the data. Once a customer contacts care, the samecustomer is unlikely to contact care if the issue is resolved. Inaddition, customers are unlikely to contact care on consecutive days, ascollected data show lapse of at least 24 hours after experiencingservice issues, as illustrated in FIG. 2B(b). It is insufficient tosimply use previous day data.

To account for behavioral aspects of care contact, maximize utilizationof samples of customers who contact care, and exploit care contactseasonality, the feature extraction module 238 in the illustratedembodiment uses a sliding window design to create the training datasetor test dataset for the prediction models 240. The sliding windowdesign, in effect, re-starts the clock when a customer calls in to care.Such calls have a seasonality to them, as illustrated by FIG. 2B(a).More calls are observed to occur on certain days of the week, forexample. For example, according to FIG. 2B(a), evidence indicates thatmore calls are received by care on Wednesday, Thursday and Friday in agiven week than on Monday. In some embodiments, if a customer callscare, the system 230 looks back from 1 to 25 days to see what thatcustomer's call history has been. Using a sliding window, the day of theweek on which the customer contacts care becomes less important. Thesliding window operates to normalize customer care information.

FIG. 2F illustrates an exemplary sliding window design for featurevectors in accordance with various aspects described herein. FIG. 2Fshows how features are aggregated to create the training dataset or testdataset using a sliding window of 7 time units, W={T-7, T-6, T-5, T- 4,T-3, T-2, T-1}. In the illustrated example, T corresponds to one day,T=0 is the day on which the customer calls care, T=−1 is the day prior,etc. The use of single day time windows may be appropriate for anaggregated system using a single system 230 and model for a large ornationwide prediction program. For smaller, distributed systems, whichhave respectively less customer data to process, shorter time windowssuch as one hour may be appropriate. The sliding window design providesinformation about what the customer's performance has been, such aswhether the customer blocked more calls than usual on the daycorresponding to T=0 or whether the customer experienced more droppedcalls than usual.

As shown in FIG. 2F, using inputs of 10 time units (T-9 to T-0), thefeature extraction module 238 creates three slices of data, slice-1using data from T-9 to T-3 using which predictions for T-2 aregenerated; slice-2, using data from T-8 to T-2, using which predictionsfor T-1 are generated; and slice-3, using data from T-7 to T-1, usingwhich predictions for T-0 are generated. Predictions are based on a setof rules about likelihood that a customer will call care, such aswhether the user device uses the Android operating system or the iOSoperating system, or whether the user make a relatively large number ofvoice calls. The rule set may be modified or adjusted over time or toreflect information learned about customers and users of thecommunication network.

This sliding window design allows the feature extraction module 238 ofthe offline phase 232 to (a) maximize the utilization of positivesamples (shortcoming (1) above), because data for each time unit is usedin multiple slices (n times with an n-unit sliding window), and (b)handle seasonality and temporal dependencies (shortcomings (2) and (3)above): The feature extraction module 238 uses care contacts made by acustomer per time unit as an additional input to the models. To addressshortcoming (4) above (unpredictable reporting time), a different timewindow is used by the online phase 234 for predicting if a customer islikely to contact care, e.g., it is predicted if a customer is likely tocontact care in the next |R| time units as a result of service issues,setting |R|={T−0, T+1, T+2, T+3, t+4}. The sliding window parameters |W|and |R| should be carefully selected to meet operational requirements.The values of the sliding window parameters |W| and |R| are discussed inmore detail below.

Let U be the stochastic process that generates data U_(i)=(U_(i,1), . .. , U_(i,n)), which are the records of the i-th user from a startingtime t0=1 until an end time tend=n, where n>|W|+|R| is assumed constant.Let N>0 be the number of users in the care logs 224 and the network logs228. Then {U_(i)}_(i=1) ^(N) is the data used to construct the trainingdataset 242. In the training dataset 242, positive cases correspond totimes when a customer did call care. Negative cases correspond to when acustomer did not call care. This may be considered to be the targetvariable. The model uses a machine learning algorithm to determine aprobability that, given a customer and given that there is 25 or 60days, for example, of call log history for the customer, what is theprobability that the customer will call care today? For the customersthat are predicted to call care, the operator of the system can take amitigation action to eliminate the need for the call by the customer,before the need arises. Again, this is done on an individual customerbasis.

The training dataset 242 may be an aggregate of the care logs 224, thecustomer account information 226 and the network logs 228. The networklogs 228 include information generated by customer care team memberswhen a customer calls into care. This may include the customer's phonenumber, the time of the call, what actions were done on the customer'sdevice, the nature of the customer's reported problem, such as a billingissue or a network issue. The customer account information 226 mayinclude static features on the customer's account such as the customer'sdevice make and model, how long the customer has had the account. Thenetwork logs 228 may include a combination of call detail records withinformation about who was called, when a call was made, if a call failedincluding if the call was dropped or blocked. Call failure informationcan be indicative of a customer more likely to complain, because of anincrease in number of calls that were blocked or dropped. Thepreprocessing operation 236 operates in part to combine the informationfrom the care logs 224, the customer account information 226 and thenetwork logs 228. In the aggregate, this may be a very large amount ofdata, from all or nearly all subscribers or customers in a network.

The sliding window feature extraction operates to normalize thepre-processed data. The effect will be that, when a customer calls carefor assistance, at day T=0, the system 230 can look back a set number ofdays, such as 60 days, to see what the customer's service has been likeon the network. This can include identifying blocked or dropped call,difficulty accessing the internet, etc. Customers that have highernumbers of failed calls are more likely to call in and complain. Thecombination of care logs 224, the customer account information 226 andthe network logs 228, for each respective customer, may be consideredthat customer's service signature. The service signature may includeadditional data or less data, depending on what information is availablefor a particular customer.

The machine learning algorithm implemented by the system, includingprediction model 240, uses the service signature of a particularcustomer to try to find other customers that have similar signatures anddraw conclusions about the likelihood of the particular customer to callcare. For example, if the customer's signature has a high number ofblocked calls, similar to other customers in the training data, and theother similar customers had a high incidence of calling care, then theprobability that the customer will call care may be relatively high, aswell. That suggests that that customer is one who should get attentionto resolve technical issues before the customer is prompted to callcare. Other factors may be determined to be important for determiningthe probability of likelihood to call, such as the nature of thecustomer's device (Android device versus iOS device) may show alikelihood to call, customers that have more voice calls in a week maybe likely to call care. The prediction model 240 may implement a seriesof rules based on empirical data or based on an analysis of pasthistory. The rules may be updated as the history develops over time. Forexample, as network topography changes as the network is built out, moreor fewer calls to care may arise. As new devices are introduced to themarket, they may cause new care issues. Similarly as old devices or oldmodels of devices, fade from use, the relative number of calls to caremay decline. The prediction model 240 may be dynamically updated fromtime to time to account for such changes.

The historical data from the care logs 224, the customer accountinformation 226 and the network logs 228 are used to train theprediction model 240. From time to time, such as daily, the system 230retrieves current network data including the care logs 224, the customeraccount information 226 and the network logs 228, such as for the mostrecent day. This retrieved data is used as test data for the predictionmodel. The test data may be supplemented with other data that may beavailable, or modified in any suitable way. The test data is applied tothe prediction model 240 and, in one embodiment, the prediction modelproduces a list of telephone numbers that have a relatively highprobability of calling into care as predictions 246. The telephonenumbers produced by the prediction model 240 serve as identifiers forcustomers that may call care. Any other suitable customer identifier maybe used and produced as predictions 246 output by the prediction model240.

The list of telephone numbers, or other identifying information, arethen used to take resolution actions 249 by the system 230. Resolutionactions 249 may include any suitable activity which may obviate a callto care by the customer associated with the identifying information.Examples of such resolution actions 249 include, for example, one ormore of the following:

Cancel Registration: Cancel/detach the UE's Serving GPRS Support Node(SGSN) and Mobility Management Entity (MME) registration to the EvolvedPacket Core Home Subscriber Server (EPC HSS).

Resend Subscriber Activation: obtain the customer's latest subscriberprofile in the billing system and update the Home Subscriber Server HomeLocation Register (HSS/HLR) with it.

Resend Over-The-air activation profile: resend the currentservices/features to the Subscriber Identify Module (SIM) card &activate the SIM if it is inactive.

Push a remote software upgrade: notifies devices (Android operatingsystem only) to upgrade to latest firmware.

Other resolution actions may be implemented. The possible resolutionactions 249 listed here are advantageous in being generally invisible tothe customer and occurring very quickly. Such network-based actionsgenerally occur in the background and the customer may see only a briefdelay. However, the noted resolution actions are generally highlyeffective for improving quality of a customer's service and thereliability of the customer's device interacting with the network. Ineffect, the machine learning algorithm of the system 230 helps identifywho are the most vulnerable customers, most likely to experience anissue. The resolution actions 249 the prophylactically takes action toreduce likelihood of the problem occurring.

The training dataset 242 D_(train) is constructed through slidingwindows of length |W|+|R| over as depicted in FIG. 2F. Morespecifically, the training dataset 242 may be considered asD_(train)={((X_(i), Z_(i)), Y_(i))}_(i), where X_(i)∈R^(|W|×p) ^(stat)is a matrix with pstat-dimensional static features of each time unit inW, Z_(i)∈R^(|W|×p) ^(dyn) is a matrix with pdyn-dimensional dynamicfeatures of each time unit in W, and Y_(i)∈{0,1}^(|R|) is a randomvariable vector containing the target label (whether or not someone willcontact the call center at each of the next |R| time units in thefuture).

Now define a function (feature generator) Φ: R^(|W|×p) ^(dyn)→R^(|W|×)p_(dyn) that takes Z_(i) of user i as input and outputs a setof features in the same space. The feature extraction module 238 thenlearns a classifier f that takes this feature matrix Φ(Z_(i)) and Xi andoutputs ŷ∈[0,1]^(|R|), and estimate of the probability

P((Y _(i))_(d)=1|X _(i) ,Z _(i))≈(ŷ _(i))_(d)=ƒ(Φ(Z _(i)),X_(i))_(d)  (1)

that user i will contact care and report an issue at the d-the time unitof a window of size |R|, i=1, . . . , N. The output of a machinelearning model is the likelihood that a particular customer will callcustomer care for assistance or with a complaint.

A straightforward approach for solving the problem described here is tobuild a binary classification model based on a combination of static anddynamic feature vectors. However, this simple binary classification maynot work well in some instances due to the challenges discussed above,such as data scale and quality, behavioral aspects of users reportingservice degradation, issue classification and fidelity of reporting, andthe correlating of network events to customer quality of experience.Different feature vector generation functions Φ from Equation 1 may bedesigned to address these challenges.

A first model to be considered is an aggregated feature model (AFM). AFMuses a combination of features extracted from (a) Static features incustomer account information 226 as shown in Table 3, i.e., X, and (b)aggregated dynamic feature vectors extracted from the network logs 228dataset and care logs 224 dataset, Table 4, i.e., Z_(i). AFM defines theΦ_(AFM) of Equation 1 as the identity function, i.e., Φ_(AMF)(α)=α whichyields

ŷ _(i) ^((AFM))=ƒ(Φ_(AFM)(Z _(i)),X _(I)),  (2)

where ŷ_(i) ^((AFM)) is the classifier described in Equation (1).

Feature importance scores generated by XGBoost were analyzed for theAFM. XGBoost is an optimized gradient boosting library available atxgboost.readthedocs.io/en/latest, for example. The top five features indecreasing order of feature importance scores are : Activation-Time,Call-Duration {T-1, T-2, T-3}, #Calls {T-1, T-2, T-3}, #Failed-Calls{T-1, T-2, T-3}, #Data-Sessions {T-1, T-2}. In case of dynamic featureswhere each feature consists of |W| entries (one entry corresponding toeach time unit in W), the value in the parenthesis { } shows the prefixof the time unit which had the highest feature importance score (sortedin decreasing order). The feature importance scores of AFM areconsistent with the observations reported above. The AFM model is veryeffective at identifying customers likely to call customer care, but maybe over-inclusive, identifying too many customers with a likelihood tocall. For example, if a threshold of 5 failed calls per month isestablished, for some customers with relatively poor network coverage, 5failed calls per month is routine whereas for customers with excellentnetwork coverage, 5 failed calls per month is an extreme, anomaloussituation likely to provoke a call to customer care.

A second model to be considered is an Individual Variations Model (IVM).IVM is designed to leverage variations in individual usage/failurepatterns. The IVM model is helpful to remove anomalous conditions and tohelp normalize the customer's experience on the network. As an example,consider a customer (in a low coverage area) who experiences an averageof n voice call failures/day. While this customer may not contact careif they continue to experience similar failures ratio per day, they arelikely to contact care when the number of failures exceeds the averagedaily failures. The IVM model is designed to detect such variations inindividual usage. The IVM model takes into account the customer's meanusage. This may include, for example, the customer's mean number ofvoice calls the customer makes, the mean number of blocked calls thecustomer experiences, or the mean number of failed calls, in apredetermined period such as one month. If the number of failed calls,for example, exceeds the mean number of calls for a period, the system230 will identify and respond to the condition.

The static features used by the IVM model are the same as the AFM model,since there are no variations in the customer account information 226dataset. Dynamic features of the IVM are created by subtracting theactual values in a given time unit from the mean value of the samefeature, where the mean value of a feature is calculated using allinstances of a given feature in the entire data D. The dynamic featuresused by IVM can be described as

Φ_(IVM)(Z _(i))=( z _(i) −Z _(i) , . . . , z _(i) −Z _(i)),  (3)

where z _(i) is a vector whose m-th component is the row-average (timeaverage) of feature (column) m in matrix Z_(i).

The output of the classifier in Equation 1, using the features createdby Φ_(IVM), is then

ŷ _(i) ^((IVM))=ƒ(Φ_(IVM)(Z _(i)),X _(i)).  (4)

The feature importance score of IVM is similar to that of AFM discussedabove.

A third model to be considered is a feedback model (FBM). Obtaining amodel with low false positives is one challenge in developing aframework for automated proactive customer issue resolution. Under somecircumstances, both AFM and IVM have relatively low precision andtherefore may result in high false positives. Thus, while overall modelaccuracy is important, minimizing false positives may also be criticalin reducing unnecessary actions and any unwanted impacts on network andcustomer devices. An ensemble model may therefore be developed that usesthe inputs from AFM and IVM to minimize the prediction errors in thesystem 230. The FBM receives the probabilities from both the AFM and theIFM and adds in a bias variable. β. The bias variable β in effect tracksperformance over time for each customer, relative to whether or not thecustomer called care. The bias variable fits a constant variable foreach customer. It serves as a normalized behavior for each customer'sbehavior. The bias variable β serves to help the model fit the databetter.

FIG. 2G illustrates an embodiment of the feedback model 256 inaccordance with various aspects described herein. The feedback model 256uses as inputs information from the aggregated feature model 258 and theindividual variations model 260. Further, the feedback model 256 mayoptionally receive bias information from bias module 262.

The feedback model 256 uses the probability values generated by AFM andIVM, i.e., ŷ_(i) ^((AFM)) and ŷ_(i) ^((IVM)) from Equation 2 andEquation 4, respectively, as inputs along with a bias variable. Recallfrom FIG. 2F that the prediction window W slides by one unit each time aprediction is made. Therefore, using the input set {T-9 to T-1} definedin FIG. 2F, there are three rounds of predictions each using slice-1,slice-2, and slice-3. Let t denote a round, which includes generating aset of predictions using slice-1 and moving the sliding window toslice-2. Given t, the output of the classifier function g is

ŷ _(i) ^((FBM))(t)=g(ŷ _(i) ^((AFM))(t),ŷ _(i) ^((IVM))(t),{circumflexover (β)}_(i)(t−1)).  (5)

where {circumflex over (β)}_(i)(t−1) is a bias given by

{circumflex over (β)}_(i)(t)=ŷ_(i) ^((FBM))(t−1)−y _(i)(t−1),

where ŷ_(i) ^((FBM)) (t−1) is the expected result from the classifierand y_(i)(t−1) is the observed Y_(i)(t) at time t−1 of user i=1, . . . ,N. The difference between ŷ_(i) ^((FBM))(t−1) and y_(i)(t−1) correspondsto whether a person called care, or not, compared with whether they wereexpected to call care. That is, for each round t, the bias parameter{circumflex over (β)}_(i)(t) stores the difference between the actuallabel Y_(i)(t) [0|1] and estimated probability ŷ_(i) ^((FBM))(t), whichis then passed as optional bias when the predictions for round t+1 aremade. In some applications, the bias variable allows the actual data tobetter fit the derived machine learning model. In other applications, nobias variable may be required, and in yet other applications, other biasvariables may be used or developed based on particular empirical data.The next section demonstrates the impact of bias parameter on theperformance of the feedback model.

The output of the feedback model 256 is a value between 0 and 1corresponding to the probability that a particular customer is likely tocall care. Before a resolution action is called or implemented, theoutput probability may be compared with a threshold. Probability valuesexceeding the threshold will be dealt with through some appropriateresolution action. The threshold may be set using any useful criteria,such as 0.7

Experimental Evaluation

A prototype of the system 230 was developed using open source libraries.In one embodiment, the system executes on a Hadoop cluster with 45 TBmemory and 9000 virtual cores. The system may be implemented usingApache Pig scripts to clean and join customer records from data locatedin both a Hadoop Distributed File System (HDFS) data lake and relationaldatabases. The feature generators and classification models may also bedeveloped in Apache Spark, using XGBoost bookmark58ensemble libraries.Other implementational equipment may be readily supplemented as well.

Evaluation methodology. Trace-driven emulations were conducted based onhistorical data to evaluate the performance of the system 230. Datasets(i.e., customer account information 226, call logs 224, and network logs228) were adapted from data of a large cellular service provider thatinclude two independent sample sets of data collected from June toDecember 2019. The training dataset contains over 500K customer devices(UIDs) and the testing dataset contains over 800K customer devices(UIDs). As shown in FIG. 2E, training and testing datasets are createdby identifying all UIDs that have contacted care during the test period,in first training set 248, and by randomly sampling the UIDs from thenetwork logs 228 dataset to form the second training set 250.

Standard statistical metrics of accuracy, precision, and recall, may becomputed using True Positives (TPs), True Negatives (TNs), FalsePositives (FPs) and False Negatives (FNs). Models may also be evaluatedin terms of the Area under receiver operating characteristic curve(AUROC). AUROC is a more robust measure of classification performance asit integrates the prediction accuracy with all possible thresholds.

Justification of design choices. First, as discussed above, the proposedmodels use data for the last |W| time units to make predictions for thenext |R| time units. Therefore, the values of |W| and |R| should becarefully selected to balance data processing overhead, desiredaccuracy, and cellular provider requirements. The impact of |W| and |R|were evaluated on the performance of all three models (AFM, IVM, andFBM). For brevity, the results of the AFM and IVM are omitted and onlyanalyze the results of the FBM model. It was found that using days asthe time unit works well, and a time unit of days is therefore used forevaluation herein. Other time units may be substituted. FIG. 2H(a) showsthat |W|=6 yields high accuracy and precision. While higher values of|W| increase precision, the processing overhead outweighs theperformance benefits. FIG. 2H(b) shows that |R|=7 outperforms othervalues. Therefore |W|=6 and |R|=7 are used for further analysis. Othervalues may be used in other embodiments.

Second, Decision Trees (DTs), Boosted Decision Trees (XGBoost), andRandom Forests (RFs) are compared. Neural networks (NN) results are noteasy to interpret, which is a key requirement for the system 230, soneural networks are not used. FIG. 2I presents the results using theidentity feature generator function Φ(α)=α. It is observed that XGBoostoutperforms both DT and RF in accuracy and precision. DT and RF yieldrelatively low precision, owing to the challenges discussed above.Therefore, XGBoost is chosen for further use in this example.

Performance of prediction models. FIG. 2J shows the accuracy, precision,and recall of AFM, IVM, and FBM models on 10 consecutive days (T 9 to T0 on the x axis). The AFM model, FIG. 2J(a), has good accuracy andrecall, but not good precision. The IVM model, FIG. 2J(b), underperformson all three metrics, compared to AFM. The FBM model, FIG, 2J(c),outperforms the other two models in all three metrics. As discussedearlier, a key motivation for the design of FBM is to reduce the numberof false positives. FIG, 2J(c) shows that the FBM model attains athreefold improvement in the precision metric, which is a measure of lowfalse positives in the classification.

All three models were also compared using the AUROC metric. Again, FBM(AUROC 0.99) outperforms AFM (AUROC 0.95) and IVM (AUROC 0.77). It isnoted that, notwithstanding the low precision, AFM has high accuracy andAUROC, due to the high imbalance of negative class labels in the inputsets.

Consider now a closer look at the impact of the optional bias parameterβ on FBM performance. FIG. 2K(a) presents FBM results without theoptional bias parameter β for a set of 6 UIDs (denoted as “A”, . . . ,“E”) for 7 consecutive days (T-7 through T-1). Predictions generated byFBM have low variance without the bias parameter β, which implies thatinput feature vectors (X) and (Z) cannot be clearly demarcated into twocategories (True/False). Updating β in Equation 5 enables the model tolearn individual customer care contact behavior. During the first roundof execution, the bias parameter β is not available as there are nostored predictions from previous rounds, and therefore FBM generates anunusable prediction which is ignored (FIG. 2J(c)). FIG. 2K(b) shows thefeature importance score for FBM. The FBM model learns from the resultsof the AFM, IVM, and the bias parameter β. The FBM model may be chosenfor field deployment and trial experiments presented in below.

System runtime scalability. The system 230 needs to process and trackeach UID experiencing a failure event during time window |W|, which insome applications requires processing hundreds of billions of datarecords per day. This is infeasible even with a distributed deploymentof the system 230. To avoid the overhead of tracking a massive volume ofUIDs, in some embodiments, the system 230 creates a list of “CandidateUIDs” by sampling network logs 228 within a specific analysis window.The sampling of the network logs 228 may include undersampling. Recallfrom FIG. 2B(b) that nearly 75% of customers call care within 21 days ofexperiencing a failure. Therefore, the system 230 may sample 21 days ofnetwork logs 228 to ensure the candidate UID list contains at least 75%of customers that are likely to call care within next R time units.Additionally, as noted in above, customers who make a large number ofvoice calls are more likely to call care, and repeat callers contributeat least 28% of care center workload, as shown in FIG. 2B(a). Thus, thesystem 230 creates a candidate list by (a) sampling voice and datafailure events in network logs 228 data, (b) sampling successful voicecall records, and (c) using care logs 224 data for customers who havecontacted care due to technical issues in the past 60 days. Table 5shows the percentage of VCR/DCR records that the system 230 needs toprocess to achieve a given coverage, where the coverage (for a given|R|) is defined as

${Coverage} = \frac{{UIDs}{with}{technical}{contact}(s){in}{the}{candidate}{set}}{{UIDs}{with}{technical}{{contact}(s)}\left( {{ground}{truth}} \right)}$

Although the percentages of UIDs sampled from each source are small, thesystem 230 needs several million UIDs to achieve a coverage of 16% to23%.

TABLE 5 Coverage of different sampling ratios for NL. VCR events DCRevents VCR volume Coverage ~0.4% ~.012% ~.01% ~16% ~0.4% ~.012% ~.04%~18% ~0.4% ~.024% ~.01% ~16% ~1.2% ~.012% ~.04% ~18% ~1.2% ~.024% ~.1%~23%   ~4% ~.012% ~.01% ~23%

The exemplary test system 230 is fully automated and performs the entireworkflow, from data analysis to triggering resolution actions, withouthuman intervention. Assuming that the system 230 needs to analyze onaverage of approximately 2 million UIDs per day, the end-to-end timetaken (analysis window size 21 days, |W|=6 days, and |R|)=7 days) is acombination of time taken in (a) candidate set generation (approximately20 mins), (b) data pre-processing (approximately 4 hours), (c) modelscoring and predictions (approximately 45 mins), and (d) resolutionaction generation (approximately 2.8 ms per UID). The entire processconsumes 76.1M GB-seconds and 2,200 k vcore-seconds of computingresources.

A test embodiment of the system 230 was deployed in a large cellularservice provider net-work in the US with a goal to understand theoverall efficacy of the system 230 and consider operation including thefollowing: (1) How effective are the predictions and actions generatedby the system 230 in resolving customer issues? (2) Did the actionstriggered by the system 230 improve QoE for cellular customers? and (3)Is it feasible for a large cellular provider to deploy an automatedframework to make online predictions and trigger resolution actions?

Field trial setup. A controlled field trial was launched in which asingle instance of the system 230 was integrated into the productionsystem of the cellular service provider. During this trial, the system230 processes the network logs 228 data of over 100 million user mobiledevices according to the procedure described for the online phase 234 inFIG. 2E using |W|=6 days and |R|=25 days, approximately 800,000 UIDswere sampled to be part of the experiment. These users were selectedaccording to historical care logs 224 and network logs 228 data in theprevious 28-day period. The system 230 is used to predict a short listof UIDs of customers who are likely to contact care due to serviceissues within the next 25 days. These UIDs were randomly split(predicted to contact care) into two groups: a control group and anexperimental group. Since the goal here is to understand the efficacy ofresolution actions generated by the system 230, only trigger resolutionactions on UIDs in the experimental group and no resolution actions aretriggered on UIDs in the control group. Note that the system 230determines the actions to take based on domain knowledge and operationalexperiences provided by the cellular service provider.

Experimental results. FIG. 2L compares results for an experimental groupand a control group. FIG. 2L presents the results of a field trial. FIG.2L(a) shows the percentage of customers in experimental and controlgroups who contacted care during the evaluation period. It may beobserved that: (1) Nearly 15% of customers predicted to contact care bythe system 230, indeed contacted care within 25 days after theyexperienced service issues, and (2) Resolution actions taken by thesystem 230 effectively reduced care contacts by 60% for customers in theexperimental group, compared to customers in the control group. In orderto understand the efficacy of the actions in improving customer QoE, thenumber of failures (data/voice) experienced by customers in theexperimental and control groups are also compared. FIG. 2L(b) thatresolution actions triggered by the system 230 effectively reduced thenumber of failures by 51% in the next 3 days after the action was taken.

During the field trial, it was observed that an embodiment of the system230 was required to process up to 1.20 billion data records per hourduring peak time. The embodiment of the system 230 consumed 6.3MGB-seconds and 297 k vcore-seconds of resources and took under 45minutes to complete.

Analytical results. Since the field trial only involves a relativelysmall percentage of the entire customer base of the cellular serviceprovider, a theoretical analysis of the controlled field trial resultsis presented here. Specifically, the hypothesis was tested that thedistributions of the control and experimental groups were equal acrossthe following two observed metrics for a period of 25 days after theresolution actions were taken: (1) the number of care contacts due toservice issues, (2) the number of voice failures and data connectivityfailures. Formally, given two populations P1 and P2 where |P|=m and|P2|=n, one can validate the null hypothesis Ho P1=P2, and the alternatehypothesis, H₁ P1≠P2, where P1 and P2 represent the observations fromthe experimental and the control distributions, respectively. Due to theindependent samples and non-normal distribution of the data, theMann-Whitney U test may be used with p-value threshold a=0.05, tovalidate the null hypothesis.

It is found that for both cases (a) the number of care contacts due toservice issues (U=21987.5, p-value=0.048), and (b) the number of voicefailures and data connectivity failures (U=22545.5, p-value=0.028), thep-value is lower than a. Therefore the null hypothesis Ho can berejected and the alternate hypothesis Hi can be accepted for both cases.That is, it may be concluded that the P1≠P2 for both (1) the number ofcare contacts due to service issues, and (2) the number of voice/dataconnectivity issues experienced by them.

Summary. The observed reduction in the number of failures along with thereduction in the number care contacts are very promising, yet theseresults are not exhaustive and more evaluation is required to betterunderstand the efficacy of the predictions models. While it isinfeasible to cover the entire customer base during the field trial, theembodiment of the system 306 has achieved significant improvements forindividual customers' QoS and QoE, and done so without having thecustomer initiate a trouble report with care. This functionality canenable operators to move from a reactive to a proactive strategy whenaddressing customers with technical issues.

Predicting when a customer will contact care. In this paper, the problemof predicting the customers who will contact care (which can be modeledas a binary classification problem) has been considered in order toproactively take action for these customers. Due to missing data andlack of well-known indicators in the network logs dataset, currentmodels do not currently predict when a customer is likely to contactcare. Instead, UIDs experiencing failures during the prediction windoware identified, and these UIDs are classified into customers who arelikely or unlikely to contact care. While it is possible that a smallpercentage of customers contact care to report issues which are notobservable from the network logs dataset, predicting the care contactbehavior of such customers is not possible without access to richbehavioral features.

Conclusions. Cellular service carriers are constantly striving toimprove the customer quality of experience. In this work, a fullyautomated framework is shown to enable carriers to shift from a reactiveto a proactive customer care strategy for technical issues. Threemachine learning-based are developed models, including a novel feedbackmodel, to predict the customers who are likely to contact customer care,using a combination of customer and network data logs. Usingpredictions, individual customer technical issues (non-outage related)are resolved by proactively invoking resolution actions, which improvesthe customer quality of experience and reduces customer care contacts.These are demonstrated by large-scale trace-driven evaluation based onreal world data collected from a major cellular service provider in theUS, as well as by field trial experiments after deploying an embodimentof the system into the cellular service provider's network.

FIG. 2M depicts an illustrative embodiment of an offline method 270 andan online method 272 in accordance with various aspects describedherein. The offline method 270 illustrates one exemplary embodiment ofan offline method for training a machine learning model for identifyingcommunication service customers most likely to call customer care. Theonline method 272 illustrates one exemplary embodiment of an onlinemethod for using a machine learning model for identifying communicationservice customers most likely to call customer care and acting toprevent the need for the customer to call care.

The offline method 270 begins with an operation of retrieving carehistory, step 273. The care history may include any useful informationabout calls to customer care by users or subscribers to thecommunication service provided by a communication service provider ornetwork operator. For example, as shown in the example of FIG. 2E, thecare history retrieved at step 273 may include call logs 224, customeraccount information 226 and network logs 228 (FIG. 2E). Some or all ofthis information may be retrieved. The retrieved information may includecurrent information or historical information collected over aparticular time frame such as one week, one month, or other.

At step 274, the offline method 270 selects one or more random useridentifiers (User IDs) and retrieves network interaction historyinformation for the customers associated with the random User IDs. Therandom user IDs retrieved at step 274 and the care history retrieved atstep 273 may be combined at preprocessing step 275. Any other suitabledata manipulation may be performed at preprocessing step 275. The resultof preprocessing step 275 is to develop for each customer a servicesignature. The combination of care logs 224, the customer accountinformation 226 and the network logs 228, for each respective customer,may be considered that customer's service signature.

At step 276, a sliding window feature extraction process is performed.This may be as illustrated in FIG. 2F, for example. Preprocessed datafrom preprocessing step 275 may be segmented in time according to asequence of time slots. In the illustrated example of FIG. 2F, ten timeslots of the preprocessed data is divided into three slices of 9 timeslots each. A first slice has time slots T-2 through T-9 with time slotT-2 corresponding to the predicted period. A second slice includes timeslots T-1 to T-8, with time slot T-1 corresponding to the predictedperiod. A third slice includes time slots t-0 through T-7, with timeslot T-0 corresponding to the predicted period. The first slice, thesecond slice and the third slice together form feature vectors oftraining data. The training data is the result of the feature extractionprocess of step 276.

At step 278, training data is developed for the prediction model and atstep 280, the prediction model is trained using the training data. Ingeneral, the care history retrieved at step 273 may include asubstantial amount of data. Accordingly, the offline process may beperformed relatively infrequently, such as one per month. In alternativeembodiments, the process of offline method 270 and online method 272 maybe distributed over a number of data centers. In such an embodiment,there will be less data to process and the offline method 270 may beperformed more frequently, such as weekly.

The online method 272 illustrates one exemplary embodiment of an onlinemethod for using a machine learning model for identifying communicationservice customers most likely to call customer care. The online method272 may be performed at any suitable frequency, such as daily. In someembodiments, the method uses a different sliding window to identifycustomers most likely to call care. For example, an embodiment uses |R|time units, where |R|={T−0, T+1, T+2, T+3, T+4 }. Any suitable window oftime units could be used instead. In an example, each time unit is oneday, but any suitable value may be used.

At step 284, a care history for the current analysis window isretrieved. In one example, the analysis window is a time duration of oneday and the care history includes all calls to customer care during thecurrent day. At step 286, User IDs in the analysis window are selected.The online method 272 needs to process and track each User IDexperiencing a failure event during each time window being analyzed. Toavoid the overhead of tracking a large number of User IDs, the onlinemethod 272 creates a list of Candidate User IDs at step 286 byunder-sampling network logs within a specific analysis window. In oneexample, the online method 272 creates a candidate list by (a) samplingvoice and data failure events in network log data, (b) samplingsuccessful voice call records, and (c) using care log data for customerswho have contacted care due to technical issues in the past 60 days. Thecandidate list is preprocessed at step 288 and at step 290, a slidingwindow feature extraction is performed, as illustrated for example inFIG. 2F. The slices of the sliding window in online method 272 mayinclude uses IR time units as noted above.

At step 292, the feature vectors for the current time slice are appliedto the prediction model developed with the training data in offlinemethod 270. At step 294, if there are more time slices to process,control returns to step 282 to select a next analysis window. If thereare no further slices to process, at step 296, the prediction isgenerated. In one example, the prediction includes a list of phonenumbers or other identifiers of individual customers likely to callcustomer care. In another example, the prediction is a probability thata selected customer is likely to call customer care. The probability mayhave a value between [0. 1]. The probability may be compared with athreshold, such as 0.50 or 0.90, received at step 296. The value of thethreshold may be set by the operator of the communication network andmay be static or dynamically adjusted, for example.

At step 298, appropriate action is taken to prevent the customer fromhaving to call customer care. For example, the device such as a cellulartelephone of the customer may be subject to a network reset toproactively correct existing network issues and to reduce theprobability that the individual customer will contact care regardingindividual service issues.

By anticipating potential service issues for individual customers inthis manner, the quality of experience (QoE) for individual customers isenhanced, increasing the customer's satisfaction with the serviceprovided. Moreover, the operational efficiency of the network operatoris enhanced by reducing the number of customer care contacts that mustoccur, along with the required subsequent investigation and mitigationprocesses. The amount of traffic and messaging in the network requiredto manage a customer issue is reduced or eliminated by reducing oreliminating a substantial portion of customer care issues. If anindividual customer is predicted by the disclosed system and method tohave an issue and to call customer care, action may be taken to resolvethe issue. This reduces the severity and degradation of customer qualityof experience and eliminates the need for the customer to contact care.

While for purposes of simplicity of explanation, the respectiveprocesses are shown and described as a series of blocks in FIG. 2M, itis to be understood and appreciated that the claimed subject matter isnot limited by the order of the blocks, as some blocks may occur indifferent orders and/or concurrently with other blocks from what isdepicted and described herein. Moreover, not all illustrated blocks maybe required to implement the methods described herein.

Referring now to FIG. 3, a block diagram is shown illustrating anexample, non-limiting embodiment of a virtualized communication network300 in accordance with various aspects described herein. In particular avirtualized communication network is presented that can be used toimplement some or all of the subsystems and functions of communicationnetwork 100, the subsystems and functions of system 230, offline method270, and online method 272 presented in FIGS. 1, 2A, 2E, 2F, 2G and 3.For example, virtualized communication network 300 can facilitate inwhole or in part collecting network log information about usage of allor a part of virtualized communications network 400 and care loginformation about customer interactions with customer carerepresentatives about device and performance issues in the virtualizedcommunication network 300, determining a likelihood that any oneindividual customer will contact customer care about an issue, andaddressing the potential issue before the issue occurs and theindividual customer contacts customer care.

In particular, a cloud networking architecture is shown that leveragescloud technologies and supports rapid innovation and scalability via atransport layer 350, a virtualized network function cloud 325 and/or oneor more cloud computing environments 375. In various embodiments, thiscloud networking architecture is an open architecture that leveragesapplication programming interfaces (APIs); reduces complexity fromservices and operations; supports more nimble business models; andrapidly and seamlessly scales to meet evolving customer requirementsincluding traffic growth, diversity of traffic types, and diversity ofperformance and reliability expectations.

In contrast to traditional network elements—which are typicallyintegrated to perform a single function, the virtualized communicationnetwork employs virtual network elements (VNEs) 330, 332, 334, etc. thatperform some or all of the functions of network elements 150, 152, 154,156, etc. For example, the network architecture can provide a substrateof networking capability, often called Network Function VirtualizationInfrastructure (NFVI) or simply infrastructure that is capable of beingdirected with software and Software Defined Networking (SDN) protocolsto perform a broad variety of network functions and services. Thisinfrastructure can include several types of substrates. The most typicaltype of substrate being servers that support Network FunctionVirtualization (NFV), followed by packet forwarding capabilities basedon generic computing resources, with specialized network technologiesbrought to bear when general purpose processors or general purposeintegrated circuit devices offered by merchants (referred to herein asmerchant silicon) are not appropriate. In this case, communicationservices can be implemented as cloud-centric workloads.

As an example, a traditional network element 150 (shown in FIG. 1), suchas an edge router can be implemented via a VNE 330 composed of NFVsoftware modules, merchant silicon, and associated controllers. Thesoftware can be written so that increasing workload consumes incrementalresources from a common resource pool, and moreover so that it'selastic: so the resources are only consumed when needed. In a similarfashion, other network elements such as other routers, switches, edgecaches, and middle-boxes are instantiated from the common resource pool.Such sharing of infrastructure across a broad set of uses makes planningand growing infrastructure easier to manage.

In an embodiment, the transport layer 350 includes fiber, cable, wiredand/or wireless transport elements, network elements and interfaces toprovide broadband access 110, wireless access 120, voice access 130,media access 140 and/or access to content sources 175 for distributionof content to any or all of the access technologies. In particular, insome cases a network element needs to be positioned at a specific place,and this allows for less sharing of common infrastructure. Other times,the network elements have specific physical layer adapters that cannotbe abstracted or virtualized, and might require special DSP code andanalog front-ends (AFEs) that do not lend themselves to implementationas VNEs 330, 332 or 334. These network elements can be included intransport layer 350.

The virtualized network function cloud 325 interfaces with the transportlayer 350 to provide the VNEs 330, 332, 334, etc. to provide specificNFVs. In particular, the virtualized network function cloud 325leverages cloud operations, applications, and architectures to supportnetworking workloads. The virtualized network elements 330, 332 and 334can employ network function software that provides either a one-for-onemapping of traditional network element function or alternately somecombination of network functions designed for cloud computing. Forexample, VNEs 330, 332 and 334 can include route reflectors, domain namesystem (DNS) servers, and dynamic host configuration protocol (DHCP)servers, system architecture evolution (SAE) and/or mobility managemententity (MME) gateways, broadband network gateways, IP edge routers forIP-VPN, Ethernet and other services, load balancers, distributers andother network elements. Because these elements don't typically need toforward large amounts of traffic, their workload can be distributedacross a number of servers — each of which adds a portion of thecapability, and overall which creates an elastic function with higheravailability than its former monolithic version. These virtual networkelements 330, 332, 334, etc. can be instantiated and managed using anorchestration approach similar to those used in cloud compute services.

The cloud computing environments 375 can interface with the virtualizednetwork function cloud 325 via APIs that expose functional capabilitiesof the VNEs 330, 332, 334, etc. to provide the flexible and expandedcapabilities to the virtualized network function cloud 325. Inparticular, network workloads may have applications distributed acrossthe virtualized network function cloud 325 and cloud computingenvironment 375 and in the commercial cloud, or might simply orchestrateworkloads supported entirely in NFV infrastructure from these thirdparty locations.

Turning now to FIG. 4, there is illustrated a block diagram of acomputing environment in accordance with various aspects describedherein. In order to provide additional context for various embodimentsof the embodiments described herein, FIG. 4 and the following discussionare intended to provide a brief, general description of a suitablecomputing environment 400 in which the various embodiments of thesubject disclosure can be implemented. In particular, computingenvironment 400 can be used in the implementation of network elements150, 152, 154, 156, access terminal 112, base station or access point122, switching device 132, media terminal 142, and/or VNEs 330, 332,334, etc. Each of these devices can be implemented viacomputer-executable instructions that can run on one or more computers,and/or in combination with other program modules and/or as a combinationof hardware and software. For example, computing environment 400 canfacilitate in whole or in part collecting network log information aboutusage of all or a part of communications network incorporating computingenvironment 400 and care log information about customer interactionswith customer care representatives about device and performance issuesin a communications network, determining a likelihood that any oneindividual customer will contact customer care about an issue, andaddressing the potential issue before the issue occurs and theindividual customer contacts customer care.

Generally, program modules comprise routines, programs, components, datastructures, etc., that perform particular tasks or implement particularabstract data types. Moreover, those skilled in the art will appreciatethat the methods can be practiced with other computer systemconfigurations, comprising single-processor or multiprocessor computersystems, minicomputers, mainframe computers, as well as personalcomputers, hand-held computing devices, microprocessor-based orprogrammable consumer electronics, and the like, each of which can beoperatively coupled to one or more associated devices.

As used herein, a processing circuit includes one or more processors aswell as other application specific circuits such as an applicationspecific integrated circuit, digital logic circuit, state machine,programmable gate array or other circuit that processes input signals ordata and that produces output signals or data in response thereto. Itshould be noted that while any functions and features described hereinin association with the operation of a processor could likewise beperformed by a processing circuit.

The illustrated embodiments of the embodiments herein can be alsopracticed in distributed computing environments where certain tasks areperformed by remote processing devices that are linked through acommunications network. In a distributed computing environment, programmodules can be located in both local and remote memory storage devices.

Computing devices typically comprise a variety of media, which cancomprise computer-readable storage media and/or communications media,which two terms are used herein differently from one another as follows.Computer-readable storage media can be any available storage media thatcan be accessed by the computer and comprises both volatile andnonvolatile media, removable and non-removable media. By way of example,and not limitation, computer-readable storage media can be implementedin connection with any method or technology for storage of informationsuch as computer-readable instructions, program modules, structured dataor unstructured data.

Computer-readable storage media can comprise, but are not limited to,random access memory (RAM), read only memory (ROM), electricallyerasable programmable read only memory (EEPROM),flash memory or othermemory technology, compact disk read only memory (CD-ROM), digitalversatile disk (DVD) or other optical disk storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devicesor other tangible and/or non-transitory media which can be used to storedesired information. In this regard, the terms “tangible” or“non-transitory” herein as applied to storage, memory orcomputer-readable media, are to be understood to exclude onlypropagating transitory signals per se as modifiers and do not relinquishrights to all standard storage, memory or computer-readable media thatare not only propagating transitory signals per se.

Computer-readable storage media can be accessed by one or more local orremote computing devices, e.g., via access requests, queries or otherdata retrieval protocols, for a variety of operations with respect tothe information stored by the medium.

Communications media typically embody computer-readable instructions,data structures, program modules or other structured or unstructureddata in a data signal such as a modulated data signal, e.g., a carrierwave or other transport mechanism, and comprises any informationdelivery or transport media. The term “modulated data signal” or signalsrefers to a signal that has one or more of its characteristics set orchanged in such a manner as to encode information in one or moresignals. By way of example, and not limitation, communication mediacomprise wired media, such as a wired network or direct-wiredconnection, and wireless media such as acoustic, RF, infrared and otherwireless media.

With reference again to FIG. 4, the example environment can comprise acomputer 402, the computer 402 comprising a processing unit 404, asystem memory 406 and a system bus 408. The system bus 408 couplessystem components including, but not limited to, the system memory 406to the processing unit 404. The processing unit 404 can be any ofvarious commercially available processors. Dual microprocessors andother multiprocessor architectures can also be employed as theprocessing unit 404.

The system bus 408 can be any of several types of bus structure that canfurther interconnect to a memory bus (with or without a memorycontroller), a peripheral bus, and a local bus using any of a variety ofcommercially available bus architectures. The system memory 406comprises ROM 410 and RAM 412. A basic input/output system (BIOS) can bestored in a non-volatile memory such as ROM, erasable programmable readonly memory (EPROM), EEPROM, which BIOS contains the basic routines thathelp to transfer information between elements within the computer 402,such as during startup. The RAM 412 can also comprise a high-speed RAMsuch as static RAM for caching data.

The computer 402 further comprises an internal hard disk drive (HDD) 414(e.g., EIDE, SATA), which internal HDD 414 can also be configured forexternal use in a suitable chassis (not shown), a magnetic floppy diskdrive (FDD) 416, (e.g., to read from or write to a removable diskette418) and an optical disk drive 420, (e.g., reading a CD-ROM disk 422 or,to read from or write to other high capacity optical media such as theDVD). The HDD 414, magnetic FDD 416 and optical disk drive 420 can beconnected to the system bus 408 by a hard disk drive interface 424, amagnetic disk drive interface 426 and an optical drive interface 428,respectively. The hard disk drive interface 424 for external driveimplementations comprises at least one or both of Universal Serial Bus(USB) and Institute of Electrical and Electronics Engineers (IEEE) 1394interface technologies. Other external drive connection technologies arewithin contemplation of the embodiments described herein.

The drives and their associated computer-readable storage media providenonvolatile storage of data, data structures, computer-executableinstructions, and so forth. For the computer 402, the drives and storagemedia accommodate the storage of any data in a suitable digital format.Although the description of computer-readable storage media above refersto a hard disk drive (HDD), a removable magnetic diskette, and aremovable optical media such as a CD or DVD, it should be appreciated bythose skilled in the art that other types of storage media which arereadable by a computer, such as zip drives, magnetic cassettes, flashmemory cards, cartridges, and the like, can also be used in the exampleoperating environment, and further, that any such storage media cancontain computer-executable instructions for performing the methodsdescribed herein.

A number of program modules can be stored in the drives and RAM 412,comprising an operating system 430, one or more application programs432, other program modules 434 and program data 436. All or portions ofthe operating system, applications, modules, and/or data can also becached in the RAM 412. The systems and methods described herein can beimplemented utilizing various commercially available operating systemsor combinations of operating systems.

A user can enter commands and information into the computer 402 throughone or more wired/wireless input devices, e.g., a keyboard 438 and apointing device, such as a mouse 440. Other input devices (not shown)can comprise a microphone, an infrared (IR) remote control, a joystick,a game pad, a stylus pen, touch screen or the like. These and otherinput devices are often connected to the processing unit 404 through aninput device interface 442 that can be coupled to the system bus 408,but can be connected by other interfaces, such as a parallel port, anIEEE 1394 serial port, a game port, a universal serial bus (USB) port,an IR interface, etc.

A monitor 444 or other type of display device can be also connected tothe system bus 408 via an interface, such as a video adapter 446. Itwill also be appreciated that in alternative embodiments, a monitor 444can also be any display device (e.g., another computer having a display,a smart phone, a tablet computer, etc.) for receiving displayinformation associated with computer 402 via any communication means,including via the Internet and cloud-based networks. In addition to themonitor 444, a computer typically comprises other peripheral outputdevices (not shown), such as speakers, printers, etc.

The computer 402 can operate in a networked environment using logicalconnections via wired and/or wireless communications to one or moreremote computers, such as a remote computer(s) 448. The remotecomputer(s) 448 can be a workstation, a server computer, a router, apersonal computer, portable computer, microprocessor-based entertainmentappliance, a peer device or other common network node, and typicallycomprises many or all of the elements described relative to the computer402, although, for purposes of brevity, only a remote memory/storagedevice 450 is illustrated. The logical connections depicted comprisewired/wireless connectivity to a local area network (LAN) 452 and/orlarger networks, e.g., a wide area network (WAN) 454. Such LAN and WANnetworking environments are commonplace in offices and companies, andfacilitate enterprise-wide computer networks, such as intranets, all ofwhich can connect to a global communications network, e.g., theInternet.

When used in a LAN networking environment, the computer 402 can beconnected to the LAN 452 through a wired and/or wireless communicationnetwork interface or adapter 456. The adapter 456 can facilitate wiredor wireless communication to the LAN 452, which can also comprise awireless AP disposed thereon for communicating with the adapter 456.

When used in a WAN networking environment, the computer 402 can comprisea modem 458 or can be connected to a communications server on the WAN454 or has other means for establishing communications over the WAN 454,such as by way of the Internet. The modem 458, which can be internal orexternal and a wired or wireless device, can be connected to the systembus 408 via the input device interface 442. In a networked environment,program modules depicted relative to the computer 402 or portionsthereof, can be stored in the remote memory/storage device 450. It willbe appreciated that the network connections shown are example and othermeans of establishing a communications link between the computers can beused.

The computer 402 can be operable to communicate with any wirelessdevices or entities operatively disposed in wireless communication,e.g., a printer, scanner, desktop and/or portable computer, portabledata assistant, communications satellite, any piece of equipment orlocation associated with a wirelessly detectable tag (e.g., a kiosk,news stand, restroom), and telephone. This can comprise WirelessFidelity (Wi-Fi) and BLUETOOTH® wireless technologies. Thus, thecommunication can be a predefined structure as with a conventionalnetwork or simply an ad hoc communication between at least two devices.

Wi-Fi can allow connection to the Internet from a couch at home, a bedin a hotel room or a conference room at work, without wires. Wi-Fi is awireless technology similar to that used in a cell phone that enablessuch devices, e.g., computers, to send and receive data indoors and out;anywhere within the range of a base station. Wi-Fi networks use radiotechnologies called IEEE 802.11 (a, b, g, n, ac, ag, etc.) to providesecure, reliable, fast wireless connectivity. A Wi-Fi network can beused to connect computers to each other, to the Internet, and to wirednetworks (which can use IEEE 802.3 or Ethernet). Wi-Fi networks operatein the unlicensed 2.4 and 5 GHz radio bands for example or with productsthat contain both bands (dual band), so the networks can providereal-world performance similar to the basic 10BaseT wired Ethernetnetworks used in many offices.

Turning now to FIG. 5, an embodiment 500 of a mobile network platform510 is shown that is an example of network elements 150, 152, 154, 156,and/or VNEs 330, 332, 334, etc. For example, platform 510 can facilitatein whole or in part collecting network log information about usage ofall or a part of a mobile communications network and care loginformation about customer interactions with customer carerepresentatives about device and performance issues in the mobilenetwork, determining a likelihood that any one individual customer willcontact customer care about an issue, and addressing the potential issuebefore the issue occurs and the individual customer contacts customercare. In one or more embodiments, the mobile network platform 510 cangenerate and receive signals transmitted and received by base stationsor access points such as base station or access point 122. Generally,mobile network platform 510 can comprise components, e.g., nodes,gateways, interfaces, servers, or disparate platforms, that facilitateboth packet-switched (PS) (e.g., internet protocol (IP), frame relay,asynchronous transfer mode (ATM)) and circuit-switched (CS) traffic(e.g., voice and data), as well as control generation for networkedwireless telecommunication. As a non-limiting example, mobile networkplatform 510 can be included in telecommunications carrier networks, andcan be considered carrier-side components as discussed elsewhere herein.Mobile network platform 510 comprises CS gateway node(s) 512 which caninterface CS traffic received from legacy networks like telephonynetwork(s) 540 (e.g., public switched telephone network (PSTN), orpublic land mobile network (PLMN)) or a signaling system #7 (SS7)network 560. CS gateway node(s) 512 can authorize and authenticatetraffic (e.g., voice) arising from such networks. Additionally, CSgateway node(s) 512 can access mobility, or roaming, data generatedthrough SS7 network 560; for instance, mobility data stored in a visitedlocation register (VLR), which can reside in memory 530. Moreover, CSgateway node(s) 512 interfaces CS-based traffic and signaling and PSgateway node(s) 518. As an example, in a 3GPP UMTS network, CS gatewaynode(s) 512 can be realized at least in part in gateway GPRS supportnode(s) (GGSN). It should be appreciated that functionality and specificoperation of CS gateway node(s) 512, PS gateway node(s) 518, and servingnode(s) 516, is provided and dictated by radio technologies utilized bymobile network platform 510 for telecommunication over a radio accessnetwork 520 with other devices, such as a radiotelephone 575.

In addition to receiving and processing CS-switched traffic andsignaling, PS gateway node(s) 518 can authorize and authenticatePS-based data sessions with served mobile devices. Data sessions cancomprise traffic, or content(s), exchanged with networks external to themobile network platform 510, like wide area network(s) (WANs) 550,enterprise network(s) 570, and service network(s) 580, which can beembodied in local area network(s) (LANs), can also be interfaced withmobile network platform 510 through PS gateway node(s) 518. It is to benoted that WANs 550 and enterprise network(s) 570 can embody, at leastin part, a service network(s) like IP multimedia subsystem (IMS). Basedon radio technology layer(s) available in technology resource(s) orradio access network 520, PS gateway node(s) 518 can generate packetdata protocol contexts when a data session is established; other datastructures that facilitate routing of packetized data also can begenerated. To that end, in an aspect, PS gateway node(s) 518 cancomprise a tunnel interface (e.g., tunnel termination gateway (TTG) in3GPP UMTS network(s) (not shown)) which can facilitate packetizedcommunication with disparate wireless network(s), such as Wi-Finetworks.

In embodiment 500, mobile network platform 510 also comprises servingnode(s) 516 that, based upon available radio technology layer(s) withintechnology resource(s) in the radio access network 520, convey thevarious packetized flows of data streams received through PS gatewaynode(s) 518. It is to be noted that for technology resource(s) that relyprimarily on CS communication, server node(s) can deliver trafficwithout reliance on PS gateway node(s) 518; for example, server node(s)can embody at least in part a mobile switching center. As an example, ina 3GPP UMTS network, serving node(s) 516 can be embodied in serving GPRSsupport node(s) (SGSN).

For radio technologies that exploit packetized communication, server(s)514 in mobile network platform 510 can execute numerous applicationsthat can generate multiple disparate packetized data streams or flows,and manage (e.g., schedule, queue, format . . . ) such flows. Suchapplication(s) can comprise add-on features to standard services (forexample, provisioning, billing, customer support . . . ) provided bymobile network platform 510. Data streams (e.g., content(s) that arepart of a voice call or data session) can be conveyed to PS gatewaynode(s) 518 for authorization/authentication and initiation of a datasession, and to serving node(s) 516 for communication thereafter. Inaddition to application server, server(s) 514 can comprise utilityserver(s), a utility server can comprise a provisioning server, anoperations and maintenance server, a security server that can implementat least in part a certificate authority and firewalls as well as othersecurity mechanisms, and the like. In an aspect, security server(s)secure communication served through mobile network platform 510 toensure network's operation and data integrity in addition toauthorization and authentication procedures that CS gateway node(s) 512and PS gateway node(s) 518 can enact. Moreover, provisioning server(s)can provision services from external network(s) like networks operatedby a disparate service provider; for instance, WAN 550 or GlobalPositioning System (GPS) network(s) (not shown). Provisioning server(s)can also provision coverage through networks associated to mobilenetwork platform 510 (e.g., deployed and operated by the same serviceprovider), such as the distributed antennas networks shown in FIG. 1(s)that enhance wireless service coverage by providing more networkcoverage.

It is to be noted that server(s) 514 can comprise one or more processorsconfigured to confer at least in part the functionality of mobilenetwork platform 510. To that end, the one or more processor can executecode instructions stored in memory 530, for example. It is should beappreciated that server(s) 514 can comprise a content manager, whichoperates in substantially the same manner as described hereinbefore.

In example embodiment 500, memory 530 can store information related tooperation of mobile network platform 510. Other operational informationcan comprise provisioning information of mobile devices served throughmobile network platform 510, subscriber databases; applicationintelligence, pricing schemes, e.g., promotional rates, flat-rateprograms, couponing campaigns; technical specification(s) consistentwith telecommunication protocols for operation of disparate radio, orwireless, technology layers; and so forth. Memory 530 can also storeinformation from at least one of telephony network(s) 540, WAN 550, SS7network 560, or enterprise network(s) 570. In an aspect, memory 530 canbe, for example, accessed as part of a data store component or as aremotely connected memory store.

In order to provide a context for the various aspects of the disclosedsubject matter, FIG. 5, and the following discussion, are intended toprovide a brief, general description of a suitable environment in whichthe various aspects of the disclosed subject matter can be implemented.While the subject matter has been described above in the general contextof computer-executable instructions of a computer program that runs on acomputer and/or computers, those skilled in the art will recognize thatthe disclosed subject matter also can be implemented in combination withother program modules. Generally, program modules comprise routines,programs, components, data structures, etc. that perform particulartasks and/or implement particular abstract data types.

Turning now to FIG. 6, an illustrative embodiment of a communicationdevice 600 is shown. The communication device 600 can serve as anillustrative embodiment of devices such as data terminals 114, mobiledevices 124, vehicle 126, display devices 144 or other client devicesfor communication via either communications network 125. For example,computing device 600 can facilitate in whole or in part collectingnetwork log information about usage of all or a part of communicationsnetwork including one or more computing devices such as computing device600 and care log information about customer interactions with customercare representatives about device and performance issues in thecommunications network, determining a likelihood that any one individualcustomer will contact customer care about an issue, and addressing thepotential issue before the issue occurs and the individual customercontacts customer care.

The communication device 600 can comprise a wireline and/or wirelesstransceiver 602 (herein transceiver 602), a user interface (UI) 604, apower supply 614, a location receiver 616, a motion sensor 618, anorientation sensor 620, and a controller 606 for managing operationsthereof. The transceiver 602 can support short-range or long-rangewireless access technologies such as Bluetooth®, ZigBee®, WiFi, DECT, orcellular communication technologies, just to mention a few (Bluetooth®and ZigBee® are trademarks registered by the Bluetooth® Special InterestGroup and the ZigBee® Alliance, respectively). Cellular technologies caninclude, for example, CDMA-1X, UMTS/HSDPA, GSM/GPRS, TDMA/EDGE, EV/DO,WiMAX, SDR, LTE, as well as other next generation wireless communicationtechnologies as they arise. The transceiver 602 can also be adapted tosupport circuit-switched wireline access technologies (such as PSTN),packet-switched wireline access technologies (such as TCP/IP, VoIP,etc.), and combinations thereof.

The UI 604 can include a depressible or touch-sensitive keypad 608 witha navigation mechanism such as a roller ball, a joystick, a mouse, or anavigation disk for manipulating operations of the communication device600. The keypad 608 can be an integral part of a housing assembly of thecommunication device 600 or an independent device operably coupledthereto by a tethered wireline interface (such as a USB cable) or awireless interface supporting for example Bluetooth®. The keypad 608 canrepresent a numeric keypad commonly used by phones, and/or a QWERTYkeypad with alphanumeric keys. The UI 604 can further include a display610 such as monochrome or color LCD (Liquid Crystal Display), OLED(Organic Light Emitting Diode) or other suitable display technology forconveying images to an end user of the communication device 600. In anembodiment where the display 610 is touch-sensitive, a portion or all ofthe keypad 608 can be presented by way of the display 610 withnavigation features.

The display 610 can use touch screen technology to also serve as a userinterface for detecting user input. As a touch screen display, thecommunication device 600 can be adapted to present a user interfacehaving graphical user interface (GUI) elements that can be selected by auser with a touch of a finger. The display 610 can be equipped withcapacitive, resistive or other forms of sensing technology to detect howmuch surface area of a user's finger has been placed on a portion of thetouch screen display. This sensing information can be used to controlthe manipulation of the GUI elements or other functions of the userinterface. The display 610 can be an integral part of the housingassembly of the communication device 600 or an independent devicecommunicatively coupled thereto by a tethered wireline interface (suchas a cable) or a wireless interface.

The UI 604 can also include an audio system 612 that utilizes audiotechnology for conveying low volume audio (such as audio heard inproximity of a human ear) and high volume audio (such as speakerphonefor hands free operation). The audio system 612 can further include amicrophone for receiving audible signals of an end user. The audiosystem 612 can also be used for voice recognition applications. The UI604 can further include an image sensor 613 such as a charged coupleddevice (CCD) camera for capturing still or moving images.

The power supply 614 can utilize common power management technologiessuch as replaceable and rechargeable batteries, supply regulationtechnologies, and/or charging system technologies for supplying energyto the components of the communication device 600 to facilitatelong-range or short-range portable communications. Alternatively, or incombination, the charging system can utilize external power sources suchas DC power supplied over a physical interface such as a USB port orother suitable tethering technologies.

The location receiver 616 can utilize location technology such as aglobal positioning system (GPS) receiver capable of assisted GPS foridentifying a location of the communication device 600 based on signalsgenerated by a constellation of GPS satellites, which can be used forfacilitating location services such as navigation. The motion sensor 618can utilize motion sensing technology such as an accelerometer, agyroscope, or other suitable motion sensing technology to detect motionof the communication device 600 in three-dimensional space. Theorientation sensor 620 can utilize orientation sensing technology suchas a magnetometer to detect the orientation of the communication device600 (north, south, west, and east, as well as combined orientations indegrees, minutes, or other suitable orientation metrics).

The communication device 600 can use the transceiver 602 to alsodetermine a proximity to a cellular, WiFi, Bluetooth®, or other wirelessaccess points by sensing techniques such as utilizing a received signalstrength indicator (RSSI) and/or signal time of arrival (TOA) or time offlight (TOF) measurements. The controller 606 can utilize computingtechnologies such as a microprocessor, a digital signal processor (DSP),programmable gate arrays, application specific integrated circuits,and/or a video processor with associated storage memory such as Flash,ROM, RAM, SRAM, DRAM or other storage technologies for executingcomputer instructions, controlling, and processing data supplied by theaforementioned components of the communication device 600.

Other components not shown in FIG. 6 can be used in one or moreembodiments of the subject disclosure. For instance, the communicationdevice 600 can include a slot for adding or removing an identity modulesuch as a Subscriber Identity Module (SIM) card or Universal IntegratedCircuit Card (UICC). SIM or UICC cards can be used for identifyingsubscriber services, executing programs, storing subscriber data, and soon.

The terms “first,” “second,” “third,” and so forth, as used in theclaims, unless otherwise clear by context, is for clarity only anddoesn't otherwise indicate or imply any order in time. For instance, “afirst determination,” “a second determination,” and “a thirddetermination,” does not indicate or imply that the first determinationis to be made before the second determination, or vice versa, etc.

In the subject specification, terms such as “store,” “storage,” “datastore,” data storage,” “database,” and substantially any otherinformation storage component relevant to operation and functionality ofa component, refer to “memory components,” or entities embodied in a“memory” or components comprising the memory. It will be appreciatedthat the memory components described herein can be either volatilememory or nonvolatile memory, or can comprise both volatile andnonvolatile memory, by way of illustration, and not limitation, volatilememory, non-volatile memory, disk storage, and memory storage. Further,nonvolatile memory can be included in read only memory (ROM),programmable ROM (PROM), electrically programmable ROM (EPROM),electrically erasable ROM (EEPROM), or flash memory. Volatile memory cancomprise random access memory (RAM), which acts as external cachememory. By way of illustration and not limitation, RAM is available inmany forms such as synchronous RAM (SRAM), dynamic RAM (DRAM),synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhancedSDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM).Additionally, the disclosed memory components of systems or methodsherein are intended to comprise, without being limited to comprising,these and any other suitable types of memory.

Moreover, it will be noted that the disclosed subject matter can bepracticed with other computer system configurations, comprisingsingle-processor or multiprocessor computer systems, mini-computingdevices, mainframe computers, as well as personal computers, hand-heldcomputing devices (e.g., PDA, phone, smartphone, watch, tabletcomputers, netbook computers, etc.), microprocessor-based orprogrammable consumer or industrial electronics, and the like. Theillustrated aspects can also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network; however, some if not allaspects of the subject disclosure can be practiced on stand-alonecomputers. In a distributed computing environment, program modules canbe located in both local and remote memory storage devices.

In one or more embodiments, information regarding use of services can begenerated including services being accessed, media consumption history,user preferences, and so forth. This information can be obtained byvarious methods including user input, detecting types of communications(e.g., video content vs. audio content), analysis of content streams,sampling, and so forth. The generating, obtaining and/or monitoring ofthis information can be responsive to an authorization provided by theuser. In one or more embodiments, an analysis of data can be subject toauthorization from user(s) associated with the data, such as an opt-in,an opt-out, acknowledgement requirements, notifications, selectiveauthorization based on types of data, and so forth.

Some of the embodiments described herein can also employ artificialintelligence (AI) to facilitate automating one or more featuresdescribed herein. The embodiments (e.g., in connection withautomatically identifying acquired cell sites that provide a maximumvalue/benefit after addition to an existing communication network) canemploy various AI-based schemes for carrying out various embodimentsthereof. Moreover, the classifier can be employed to determine a rankingor priority of each cell site of the acquired network. A classifier is afunction that maps an input attribute vector, x=(x1, x2, x3, x4, . . . ,xn), to a confidence that the input belongs to a class, that is,f(x)=confidence (class). Such classification can employ a probabilisticand/or statistical-based analysis (e.g., factoring into the analysisutilities and costs) to determine or infer an action that a user desiresto be automatically performed. A support vector machine (SVM) is anexample of a classifier that can be employed. The SVM operates byfinding a hypersurface in the space of possible inputs, which thehypersurface attempts to split the triggering criteria from thenon-triggering events. Intuitively, this makes the classificationcorrect for testing data that is near, but not identical to trainingdata. Other directed and undirected model classification approachescomprise, e.g., naïve Bayes, Bayesian networks, decision trees, neuralnetworks, fuzzy logic models, and probabilistic classification modelsproviding different patterns of independence can be employed.Classification as used herein also is inclusive of statisticalregression that is utilized to develop models of priority.

As will be readily appreciated, one or more of the embodiments canemploy classifiers that are explicitly trained (e.g., via a generictraining data) as well as implicitly trained (e.g., via observing UEbehavior, operator preferences, historical information, receivingextrinsic information). For example, SVMs can be configured via alearning or training phase within a classifier constructor and featureselection module. Thus, the classifier(s) can be used to automaticallylearn and perform a number of functions, including but not limited todetermining according to predetermined criteria which of the acquiredcell sites will benefit a maximum number of subscribers and/or which ofthe acquired cell sites will add minimum value to the existingcommunication network coverage, etc.

As used in some contexts in this application, in some embodiments, theterms “component,” “system” and the like are intended to refer to, orcomprise, a computer-related entity or an entity related to anoperational apparatus with one or more specific functionalities, whereinthe entity can be either hardware, a combination of hardware andsoftware, software, or software in execution. As an example, a componentmay be, but is not limited to being, a process running on a processor, aprocessor, an object, an executable, a thread of execution,computer-executable instructions, a program, and/or a computer. By wayof illustration and not limitation, both an application running on aserver and the server can be a component. One or more components mayreside within a process and/or thread of execution and a component maybe localized on one computer and/or distributed between two or morecomputers. In addition, these components can execute from variouscomputer readable media having various data structures stored thereon.The components may communicate via local and/or remote processes such asin accordance with a signal having one or more data packets (e.g., datafrom one component interacting with another component in a local system,distributed system, and/or across a network such as the Internet withother systems via the signal). As another example, a component can be anapparatus with specific functionality provided by mechanical partsoperated by electric or electronic circuitry, which is operated by asoftware or firmware application executed by a processor, wherein theprocessor can be internal or external to the apparatus and executes atleast a part of the software or firmware application. As yet anotherexample, a component can be an apparatus that provides specificfunctionality through electronic components without mechanical parts,the electronic components can comprise a processor therein to executesoftware or firmware that confers at least in part the functionality ofthe electronic components. While various components have beenillustrated as separate components, it will be appreciated that multiplecomponents can be implemented as a single component, or a singlecomponent can be implemented as multiple components, without departingfrom example embodiments.

Further, the various embodiments can be implemented as a method,apparatus or article of manufacture using standard programming and/orengineering techniques to produce software, firmware, hardware or anycombination thereof to control a computer to implement the disclosedsubject matter. The term “article of manufacture” as used herein isintended to encompass a computer program accessible from anycomputer-readable device or computer-readable storage/communicationsmedia. For example, computer readable storage media can include, but arenot limited to, magnetic storage devices (e.g., hard disk, floppy disk,magnetic strips), optical disks (e.g., compact disk (CD), digitalversatile disk (DVD)), smart cards, and flash memory devices (e.g.,card, stick, key drive). Of course, those skilled in the art willrecognize many modifications can be made to this configuration withoutdeparting from the scope or spirit of the various embodiments.

In addition, the words “example” and “exemplary” are used herein to meanserving as an instance or illustration. Any embodiment or designdescribed herein as “example” or “exemplary” is not necessarily to beconstrued as preferred or advantageous over other embodiments ordesigns. Rather, use of the word example or exemplary is intended topresent concepts in a concrete fashion. As used in this application, theterm “or” is intended to mean an inclusive “or” rather than an exclusive“or”. That is, unless specified otherwise or clear from context, “Xemploys A or B” is intended to mean any of the natural inclusivepermutations. That is, if X employs A; X employs B; or X employs both Aand B, then “X employs A or B” is satisfied under any of the foregoinginstances. In addition, the articles “a” and “an” as used in thisapplication and the appended claims should generally be construed tomean “one or more” unless specified otherwise or clear from context tobe directed to a singular form.

Moreover, terms such as “user equipment,” “mobile station,” “mobile,”subscriber station,” “access terminal,” “terminal,” “handset,” “mobiledevice” (and/or terms representing similar terminology) can refer to awireless device utilized by a subscriber or user of a wirelesscommunication service to receive or convey data, control, voice, video,sound, gaming or substantially any data-stream or signaling-stream. Theforegoing terms are utilized interchangeably herein and with referenceto the related drawings.

Furthermore, the terms “user,” “subscriber,” “customer,” “consumer” andthe like are employed interchangeably throughout, unless contextwarrants particular distinctions among the terms. It should beappreciated that such terms can refer to human entities or automatedcomponents supported through artificial intelligence (e.g., a capacityto make inference based, at least, on complex mathematical formalisms),which can provide simulated vision, sound recognition and so forth.

As employed herein, the term “processor” can refer to substantially anycomputing processing unit or device comprising, but not limited tocomprising, single-core processors; single-processors with softwaremultithread execution capability; multi-core processors; multi-coreprocessors with software multithread execution capability; multi-coreprocessors with hardware multithread technology; parallel platforms; andparallel platforms with distributed shared memory. Additionally, aprocessor can refer to an integrated circuit, an application specificintegrated circuit (ASIC), a digital signal processor (DSP), a fieldprogrammable gate array (FPGA), a programmable logic controller (PLC), acomplex programmable logic device (CPLD), a discrete gate or transistorlogic, discrete hardware components or any combination thereof designedto perform the functions described herein. Processors can exploitnano-scale architectures such as, but not limited to, molecular andquantum-dot based transistors, switches and gates, in order to optimizespace usage or enhance performance of user equipment. A processor canalso be implemented as a combination of computing processing units.

As used herein, terms such as “data storage,” data storage,” “database,”and substantially any other information storage component relevant tooperation and functionality of a component, refer to “memorycomponents,” or entities embodied in a “memory” or components comprisingthe memory. It will be appreciated that the memory components orcomputer-readable storage media, described herein can be either volatilememory or nonvolatile memory or can include both volatile andnonvolatile memory.

What has been described above includes mere examples of variousembodiments. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing these examples, but one of ordinary skill in the art canrecognize that many further combinations and permutations of the presentembodiments are possible. Accordingly, the embodiments disclosed and/orclaimed herein are intended to embrace all such alterations,modifications and variations that fall within the spirit and scope ofthe appended claims. Furthermore, to the extent that the term “includes”is used in either the detailed description or the claims, such term isintended to be inclusive in a manner similar to the term “comprising” as“comprising” is interpreted when employed as a transitional word in aclaim.

In addition, a flow diagram may include a “start” and/or “continue”indication. The “start” and “continue” indications reflect that thesteps presented can optionally be incorporated in or otherwise used inconjunction with other routines. In this context, “start” indicates thebeginning of the first step presented and may be preceded by otheractivities not specifically shown. Further, the “continue” indicationreflects that the steps presented may be performed multiple times and/ormay be succeeded by other activities not specifically shown. Further,while a flow diagram indicates a particular ordering of steps, otherorderings are likewise possible provided that the principles ofcausality are maintained.

As may also be used herein, the term(s) “operably coupled to”, “coupledto”, and/or “coupling” includes direct coupling between items and/orindirect coupling between items via one or more intervening items. Suchitems and intervening items include, but are not limited to, junctions,communication paths, components, circuit elements, circuits, functionalblocks, and/or devices. As an example of indirect coupling, a signalconveyed from a first item to a second item may be modified by one ormore intervening items by modifying the form, nature or format ofinformation in a signal, while one or more elements of the informationin the signal are nevertheless conveyed in a manner than can berecognized by the second item. In a further example of indirectcoupling, an action in a first item can cause a reaction on the seconditem, as a result of actions and/or reactions in one or more interveningitems.

Although specific embodiments have been illustrated and describedherein, it should be appreciated that any arrangement which achieves thesame or similar purpose may be substituted for the embodiments describedor shown by the subject disclosure. The subject disclosure is intendedto cover any and all adaptations or variations of various embodiments.Combinations of the above embodiments, and other embodiments notspecifically described herein, can be used in the subject disclosure.For instance, one or more features from one or more embodiments can becombined with one or more features of one or more other embodiments. Inone or more embodiments, features that are positively recited can alsobe negatively recited and excluded from the embodiment with or withoutreplacement by another structural and/or functional feature. The stepsor functions described with respect to the embodiments of the subjectdisclosure can be performed in any order. The steps or functionsdescribed with respect to the embodiments of the subject disclosure canbe performed alone or in combination with other steps or functions ofthe subject disclosure, as well as from other embodiments or from othersteps that have not been described in the subject disclosure. Further,more than or less than all of the features described with respect to anembodiment can also be utilized.

What is claimed is:
 1. A device, comprising: a processing system including a processor; and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising: retrieving historical call information for a plurality of customers, including a selected customer, of a communication service provider; training a machine learning model using training data based on the historical call information, wherein the training the machine learning model comprises defining a plurality of slices, each respective slice of the plurality of slices segmented to include a plurality of time units, each respective time unit including associated historical call information, wherein at least some respective time units of the respective slice of the plurality of slices overlap in time to reuse the associated historical call information to form the training data for the machine learning model; and applying current call information for the selected customer to the machine learning model to determine a likelihood that the selected customer will require customer care assistance by the communication service provider.
 2. The device of claim 1, wherein the operations further comprise: responsive to the likelihood that the selected customer will require customer care assistance exceeding a predetermined threshold, initiating a corrective action on a communication network of the communication service provider to reduce the likelihood that the selected customer will require customer care assistance.
 3. The device of claim 1, wherein the operations further comprise: identifying a customer device used by the selected customer to access a communication network of the communication service provider; and initiating, by the communication network, a cancelation of a registration of the customer device with the communication network of the communication service provider as a correction action.
 4. The device of claim 1, wherein the operations further comprise: applying a sliding window feature extraction process to the historical call information to remove seasonality effects in the historical call information.
 5. The device of claim 4, wherein the operations further comprise: applying a second sliding window feature extraction process to the current call information before applying the current call information to the machine learning model, wherein the second sliding window feature extraction process differs from the sliding window feature extraction process.
 6. The device of claim 1, wherein retrieving the historical call information comprises: retrieving a care log dataset, including receiving information about a plurality of customer care calls including information about a customer associated with a respective customer care call, information about a customer care issue associated with the respective customer care call and information about actions taken for resolving the customer care issue associated with the respective customer care call and retrieving a network log dataset, including receiving information about a plurality of data communication sessions including information about a customer associated with a respective data communication session and information about a cause of termination of the respective data communication session.
 7. The device of claim 1, wherein the operations further comprise: applying the current call information for the selected customer to an aggregated features model (AFM) to determine a first likelihood that the selected customer will require customer care assistance.
 8. The device of claim 7, wherein the operations further comprise: applying the current call information for the selected customer to an individual variations model (IVM) to determine a second likelihood that the selected customer will require customer care assistance.
 9. The device of claim 8, wherein the operations further comprise: applying the first likelihood that the selected customer will require customer care assistance and the second likelihood that the selected customer will require customer care assistance to a feedback model (FBM); and receiving from the feedback model a value corresponding to the likelihood that the selected customer will require customer care assistance.
 10. The device of claim 1, wherein the device comprises one device of a plurality of similar devices, each respective device of the plurality of similar devices is dedicated to a respective unique portion of a communication network of the communication service provider.
 11. A non-transitory, computer-readable medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations, the operations comprising: receiving log information about communications between a communication network of a communication service provider and a plurality of customers of the communication service provider; extracting feature vectors from the log information including defining, using a sliding window, a plurality of slices, each respective slice of the plurality of slices segmented to include a plurality of time units, each respective time unit including associated log information, wherein at least some respective time units of the each respective slice of the plurality of slices overlap in time to reuse the associated log information to form the feature vectors; providing call information to a machine learning model, wherein the providing call information comprises providing information about current calls for a selected customer of the plurality of customers for an analysis window; and receiving, from the machine learning model, information about a likelihood that the selected customer will require customer care assistance by the communication service provider.
 12. The non-transitory, computer-readable medium of claim 11, wherein the operations further comprise: training the machine learning model using the feature vectors.
 13. The non-transitory, computer-readable medium of claim 11, wherein the providing information about current calls for a selected customer comprises: providing current voice call information for the selected customer; and providing current data call information for the selected customer for the analysis window.
 14. The non-transitory, computer-readable medium of claim 11, wherein the operations further comprise: responsive to the likelihood that the selected customer will require customer care assistance exceeding a predetermined threshold, initiating a corrective action on the communication network of the communication service provider to reduce the likelihood that the selected customer will require customer care assistance.
 15. The non-transitory, computer-readable medium of claim 14, wherein the operations further comprise: based on the likelihood that the selected customer will require customer care assistance, providing a telephone number of the selected customer; based on the telephone number of the selected customer, identifying a customer device used by the selected customer to access the communication network of the communication service provider; and initiating, by the communication network, a registration cancelation of the customer device and the communication network as a correction action.
 16. The non-transitory, computer-readable medium of claim 12, wherein the operations further comprise: providing the information about current calls for the selected customer of the plurality of customers to a first model to generate a first probability, wherein the first model aggregates features of the current call information; providing the information about current calls for the selected customer of the plurality of customers to a second model to generate a second probability, wherein the second model averages usage of the communication network of the communication service provider by the selected customer; and combining the first probability and the second probability to produce the information about a likelihood that the selected customer will require customer care assistance.
 17. A method, comprising: automatically detecting, by a first processing system including a first processor, a service issue of a telecommunication system that may impact a customer of a telecommunication service provider of the telecommunication system; forming, by a second processing system including a second processor, training data for a machine learning model, the forming the training data comprising defining, using a sliding window, a plurality of slices, each respective slice of the plurality of slices segmented to include a plurality of time units, each respective time unit including associated historical call information of the telecommunication system, wherein at least some respective time units of the each respective slice of the plurality of slices overlap in time to reuse the associated historical call information; training, by the second processing system, the machine learning model using the training data; predicting, by the second processing system implementing the machine learning model, a future customer care interaction by the customer as a result of the service issue; initiating, by the first processing system, a resolution action of the telecommunication system for the service issue, wherein the initiating is responsive to the predicting the future customer care interaction; and modifying a component of the telecommunication system to improve efficiency of operation of the telecommunication system by reducing a number of customer care contacts by customers of the telecommunication service provider.
 18. The method of claim 17, further comprising: retrieving, by the second processing system, historical data about operation of the telecommunication system, the historical data comprising the associated historical call information; extracting, by the second processing system, feature data from the historical data to generate feature vectors; and training, by the second processing system, the machine learning model using the feature vectors.
 19. The method of claim 18, wherein the predicting a further customer care interaction comprises: determining, by the second processing system, a likelihood that the customer will initiate the future customer care interaction using the machine learning model.
 20. The method of claim 19, wherein the first processing system and the second processing system are a same processing system. 